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Abstract 

The  Department  of  Defense  Architectural  Framework  (DoDAF)  describes  29 
distinct  views  but  offers  limited  guidance  on  view  selection  to  meet  system  needs.  This 
research  extends  the  Value-Driven  Enterprise  Architecture  Score  (VDEA-Score)  from  a 
descriptive,  evaluation  protocol  toward  a  prescriptive  one  by  evaluating  each  DoDAF 
view  and  its  contribution  to  the  overall  objective  of  the  completed  architecture.  This 
extension  of  VDEA  is  referred  to  as  VDEA-Development  Goals  (VDEA-DG).  The 
program  manager  or  other  decision-makers  may  use  this  insight  to  justify  the  allocation 
of  resources  to  the  development  of  specific  architecture  views  considered  to  provide 
maximum  value.  This  research  provides  insight  into  the  Joint  Capabilities  Integration 
and  Development  System  (JCIDS)  process  and  policy  requirements.  Existing  guidance 
of  a  static  list  of  views  prior  to  DoD  milestone  approval  detracts  from  the  creation  of  vital 
architecture  for  system  success.  This  research  shows  overlap  between  the  most  important 
views  for  the  considered  architecture  project  and  the  JCIDS  requirements  and  identifies 
areas  for  JCIDS  policy  improvement.  This  research  also  identifies  areas  where  DoDAF 
does  not  directly  support  the  creation  of  capabilities.  With  additional  information  on  the 
resources  required  for  creating  individual  views,  the  tool  could  be  expanded  to  identify  an 
optimal  build  sequence  given  resource  constraints. 
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METHODOLOGY  FOR  VALUE-DRIVEN  ENTERPRISE  ARCHITECTURE 


DEVELOPMENT  GOALS:  APPLICATION  TO  DODAF  FRAMEWORK 


Chapter  1.  Introduction 


The  Department  of  Defense  Architecture  Framework  (DoDAF)  Version  1.5  is 
intended  to  be  the  guide  for  all  architecture  development  in  the  Department  of  Defense 
(DoD)  (Department  of  Defense,  2007).  With  a  total  of  29  views  described  in  DoDAF, 
systems  architects  are  presented  with  a  dynamic  tool  kit  from  which  to  draw  when 
making  decisions  about  how  to  depict  their  system.  Although  the  Joint  Capabilities 
Integration  and  Development  System  (JCIDS)  prescribes  the  use  of  11  views  in  the 
system  acquisition  process,  there  is  little  guidance  as  to  which  views  should  be  developed 
(Chairman  of  the  Joint  Chiefs  of  Staff,  2007).  However,  development  of  all  29  views  is 
impractical  and  it  is  doubtful  that  such  an  effort  would  contribute  significantly  to  system 
development.  Therefore,  some  criteria  are  needed  to  decide  which  views  provide  the 
maximum  value  and  justify  the  expenditure  of  time,  effort,  and  money  to  produce.  This 
thesis  presents  a  methodology  for  making  those  types  of  decisions  about  DoDAF  view 
creation. 

1.1.  Background 

Force  protection  operations  are  currently  disjointed  and  lack  a  common  concept 
of  operations  (CONOPS)  to  integrate  services,  DoD  agencies,  and  combatant  commands 
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for  the  purpose  of  providing  protection  for  U.S.  forces  from  deployment  origin  through 
employment  and  redeployment  (Protection  Assessment  Branch,  Joint  Staff,  2004).  The 
lack  of  a  common  framework  creates  the  potential  for  gaps  in  capabilities  and  an  inability 
to  put  forth  a  united  effort  toward  the  common  goal  of  force  protection.  Differing 
standards  and  procedures  among  services  can  also  create  gaps  and  hinder  cooperation 
under  the  joint  operations  of  a  combatant  commander.  Furthermore,  lack  of  consistency 
between  strategies  at  deployment  origin  and  employment  location  can  cause 
vulnerabilities.  Such  a  compartmentalized  approach  to  force  protection  creates  an 
environment  that  is  likely  to  discourage  sharing  of  critical  information,  which  leads  to 
low  situational  awareness  and  manpower  intensive,  reactionary  responses  to  threats  and 
attacks  (Protection  Assessment  Branch,  Joint  Staff,  2004). 

The  Joint  Vision  2020  (2000)  outlines  a  strategic  vision  for  joint  operations 
including  “full  dimensional  protection”  (Chairman  of  the  Joint  Chiefs  of  Staff,  2000,  p. 

3).  By  conducting  force  protection  operations  in  a  joint  environment  and  combining  the 
core  competencies  of  individual  services,  the  combatant  commander  has  more  options 
and  greater  flexibility.  In  order  to  accomplish  the  strategic  vision  outlined  by  the  Joint 
Vision  2020,  the  military  needs  to  transition  to  a  joint  force,  to  include  “intellectually, 
operationally,  organizationally,  doctrinally,  and  technically”  (Chairman  of  the  Joint 
Chiefs  of  Staff,  2000,  p.  2). 

The  transition  to  joint  force  protection  requires  a  common  description  that  meets 
the  needs  of  all  services.  The  Joint  Force  Protection  Advanced  Security  System 
(JFPASS)  initiative  seeks  to  describe  a  Joint  Force  Protection  Concept  of  Operations  (JFP 
CONOPS)  using  an  enterprise  architecture  based  on  the  DoDAF.  The  stakeholders  in  a 
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JFP  CONOPS  span  four  services  and  multiple  functions  within  each  service.  The 
challenge  of  creating  the  JFPASS  architecture  requires  senior  decision-makers  to  balance 
the  wide  array  of  values  and  concerns  that  these  stakeholders  have.  Value-Focused 
Thinking  (VFT)  is  a  decision  analysis  tool  uniquely  suited  to  helping  decision-makers 
strike  the  appropriate  balance  between  objectives  and  facilitates  communication  in  a 
multiple  stakeholder  decision  problem  (Keeney,  1994). 

Existing  techniques  for  architecture  evaluation  are  limited  and  focus  mainly  on 
single  aspects  of  architecture  such  as  interoperability  (Ford,  Colombi,  Graham,  & 

Jacques,  2007).  For  example,  the  Enterprise  Architecture  Scorecard  provides  a  subjective 
guide  for  evaluating  an  architecture  for  completeness  (Institute  For  Enterprise 
Architecture  Developments,  2007).  However,  there  does  not  appear  to  be  any 
methodology  for  the  evaluation  of  the  importance  of  an  architectural  view  to  the  overall 
architecture.  Therefore,  this  research  will  seek  to  fill  that  gap  by  demonstrating  a 
methodology  for  guiding  the  selection  of  views  from  the  DoDAF  to  achieve  the  desired 
value  from  the  resulting  architecture.  This  will  be  accomplished  by  extending  the  Value- 
Driven  Enterprise  Architecture  (VDEA)  score,  developed  by  Cotton  and  Haase  (2009) 
and  Mills  (2009),  to  the  planning  and  development  of  architecture. 

1.2.  Research  Questions 

This  research  will  link  the  individual  views  as  described  by  the  DoDAF  to  the 
value  they  can  contribute  to  the  objectives  of  the  JFPASS  architecture  effort.  By  linking 
views  to  the  lowest-level  objectives  of  an  objective  hierarchy  elicited  using  VFT,  each 
view  will  be  evaluated  on  its  contribution  to  the  overall  objective.  This  research  will 
answer  the  following  questions: 
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1.  What  DoDAF  views  are  the  most  important  to  the  JFPASS  architecture? 

2.  What  DoDAF  views  should  be  built  based  on  the  overall  objective  of  the  JFPASS 
architecture? 

3.  Which  if  any  JCIDS  required  views  are  emphasized  by  the  values  associated  with 
the  JFPASS  architecture? 

4.  Which  if  any  views  that  are  important  to  the  JFPASS  architecture  are  not  required 
by  JCIDS? 

5.  Based  on  the  suggested  network  diagram  from  the  DoDAF  Deskbook 
(Department  of  Defense,  2003),  in  what  order  should  the  views  be  created  to  most 
rapidly  increase  the  usefulness  of  the  JFPASS  Architecture? 

1.3.  Methodology 

The  VDEA  methodology  provides  a  means  for  developing  a  weighted  value 
hierarchy  to  describe  the  values  associated  with  an  architecture  and  how  those  objectives 
contribute  to  the  fundamental  objective  as  collectively  defined  by  the  stakeholders. 
VDEA  uses  the  value  hierarchy  to  provide  a  means  for  evaluating  a  single  architecture’s 
progress  toward  meeting  the  fundamental  objective.  This  research  proposes  an  extension 
to  VDEA  to  evaluate  individual  views  and  determine  the  contribution  of  views  or  set  of 
views  to  achieving  the  fundamental  objective. 

The  value  hierarchy  and  associated  measures  will  be  used  to  create  a  “measures- 
by-views”  matrix.  Each  cell  in  the  matrix  will  represent  the  relationship  between  a 
particular  view  and  a  particular  measure,  by  which  the  architecture  is  being  scored.  This 
process  is  similar  to  the  creation  of  an  “ends-by-ways”  matrix  (RAND  Arroyo  Center, 
2006)  or  a  “cause-and-effect”  matrix  (Tague,  2005).  The  “measures-by-views”  matrix 
will  identify  the  relationship  between  measures  and  views  and  numerically  describe  those 
relationships;  when  combined  with  the  weighted  value  hierarchy,  it  will  enable  the 
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calculation  of  a  score  for  single  views  and  combinations  of  views.  Using  the  ranked 
order  of  views  and  the  suggested  network  diagram  from  the  DoDAF  deskbook  (2003),  a 
recommendation  of  which  views  to  create  and  in  which  order  can  be  generated. 

1.4.  Assumptions  and  Limitations 

This  research  will  seek  to  use  global  evaluation  measure  weights  from  a  value 
hierarchy  as  a  proxy  to  ascertain  the  importance  of  particular  views  to  the  completed 
architecture.  In  doing  this,  the  value  created  by  interdependencies  between  views  was 
not  considered;  instead,  the  importance  of  each  view  was  assumed  to  be  linearly  additive 
as  determined  by  the  evaluation  measure  weight.  Additionally,  multiple  views  can 
address  a  single  measure  but  only  one  may  be  required.  In  the  case  where  two  or  more 
views  address  a  single  measure,  this  research  did  not  distinguish  between  them  as  to  their 
efficacy  in  doing  so. 

Assuming  sufficient  quality  architecture  views,  measures  aimed  at  evaluating 
correctness  or  compliance  with  standards  were  not  evaluated.  It  was  also  assumed  that 
the  system  being  described  is  the  correct  system  for  the  purpose  being  considered. 
Combining  these  assumptions,  any  view  added  to  the  architecture  can  only  improve  the 
architecture  by  further  describing  the  system. 

This  research  sought  to  demonstrate  a  methodology  for  evaluating  the  DoDAF 
views;  in  doing  so,  a  weighted  value  hierarchy  was  used  that  was  specific  to  a  Joint  Force 
Protection  Concept  of  Operations.  Application  of  this  methodology  to  other  architecture 
efforts  will  require  a  value  hierarchy  that  is  applicable  to  that  system.  Creation  of  a  value 
hierarchy  can  be  a  time  consuming  endeavor  but  if  done  correctly  can  provide  benefits 
beyond  the  applications  presented  here.  The  value  hierarchy  can  also  be  used  for 
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evaluation  of  the  architecture  during  development  to  measure  progress  as  is  presented  by 
Cotton  and  Haase  (2009);  Mills  (2009);  Cotton,  Haase,  Havlicek,  and  Thai  (2009);  and 
Mills,  Osgood,  Thai,  and  Havlicek  (2009). 

1.5.  Significance  of  Study 

This  research  demonstrates  an  extension  of  the  VDEA  score  methodology  to  the 
evaluation  of  individual  architecture  views.  The  methodology  presented  here  provides  a 
means  for  evaluating  individual  DoDAF  views  and  their  contribution  to  the  overall 
architecture.  This  extension  is  referred  to  as  VDEA-Development  Goals  (VDEA-DG). 
This  provides  the  program  manager  or  other  decision-makers  with  a  convenient  tool  for 
resource  allocation  to  the  development  of  views.  With  additional  information  on  the 
resources  required  for  creating  individual  views,  the  tool  could  be  expanded  to  identify  an 
optimal  set  of  views  in  a  resource-constrained  environment. 

Beyond  the  system  development  program,  this  research  provides  insight  into  the 
JCIDS  process  and  the  views  it  requires.  The  static  list  of  views  required  for  JCIDS 
milestone  approvals  detracts  from  the  creation  of  architecture  for  the  purpose  of 
improving  an  acquisition  program.  With  a  limited  amount  of  resources  to  devote  to 
architecture  development,  required  views  that  add  limited  value  to  the  program  take 
resources  away  from  the  creation  of  views  that  are  more  important  for  the  objectives  of 
the  architecture.  The  methodology  presented  in  this  research  provides  a  means  for 
justifying  the  expenditure  of  resources  on  the  most  important  views. 

Finally,  this  research  will  provide  insight  into  the  DoDAF  and  JCIDS  and  their 
consequences  for  value  creation  in  architecture.  An  examination  of  the  ways  in  which 
the  DoDAF  views  contribute  to  the  objectives  of  architecture  provides  a  critique  of  the 
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entire  DoDAF.  Comparing  the  results  of  this  research  to  JCIDS  requirements  for 
architecture  can  identify  opportunities  for  increasing  JCIDS  support  for  creation  of 
architecture  to  improve  decision-making. 

1.6.  Overview  of  Remaining  Chapters 

The  remaining  chapters  introduce  the  concepts  necessary  for  understanding  this 
research,  present  the  methodology  used  and  the  subsequent  results,  and  draw  conclusions 
and  recommendations.  Chapter  2  provides  a  review  of  joint  force  protection  and  system 
architecture,  particularly  the  DoDAF.  It  also  discusses  the  evaluation  of  architecture  and 
VFT,  as  well  as  previous  research  pertinent  to  the  current  study.  The  methodology  used 
to  create  a  proxy  for  importance  and  develop  a  tool  for  evaluating  architecture  views  is 
discussed  in  Chapter  3.  Chapter  4  describes  how  the  methodology  was  operationalized 
and  presents  the  results  and  analysis.  Finally,  Chapter  5  interprets  the  results  of  the 
analysis,  draws  conclusions,  and  makes  recommendations  for  further  research. 
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Chapter  2.  Literature  Review 


This  chapter  introduces  the  concept  of  joint  force  protection  and  its  importance  to 
joint  operations.  It  also  provides  an  overview  of  enterprise  architecture  and  the 
Department  of  Defense  Architectural  Framework  (DoDAF)  version  1.5  (2007)  and  how  it 
is  being  applied  to  joint  force  protection.  Existing  literature  on  the  evaluation  of 
architecture  is  explored  followed  by  a  discussion  of  the  Value-Focused  Thinking  (VFT) 
methodology  and  management  tools  for  measuring  performance.  The  chapter  concludes 
with  examples  of  research  that  measured  value  contribution  of  activities  and  resources 
towards  a  set  of  objectives. 

2.1.  Joint  Force  Protection 

The  flexibility  and  synergy  of  United  States  military  joint  operations  is  important 
when  engaging  an  adaptive  enemy.  The  ability  to  protect  the  joint  force  by  countering 
asymmetrical  threats  aimed  at  degrading  capabilities  and  the  will  to  fight  is  necessary  to 
be  effective  in  warfare.  This  need  for  security  in  a  joint  operations  environment  is  what 
necessitates  the  implementation  of  a  joint  force  protection  concept  of  operations 
(Chairman  of  the  Joint  Chiefs  of  Staff,  2004). 

The  Department  of  Defense’s  (DoD)  doctrine  on  joint  operations  provides 
guidance  to  joint  commanders  in  the  implementation  of  joint  operations  and  describes 
force  protection  as, 

Force  protection  includes  preventive  measures  taken  to  mitigate  hostile  actions 
against  DoD  personnel  (to  include  family  members),  resources,  facilities,  and 
critical  information.  These  actions  conserve  the  force’s  fighting  potential  so  it 
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can  be  applied  at  the  decisive  time  and  place  and  incorporates  the  integrated  and 
synchronized  offensive  and  defensive  measures  to  enable  the  effective 
employment  of  the  joint  force  while  degrading  opportunities  for  the  adversary... 
Force  protection  is  achieved  through  the  tailored  selection  and  application  of 
multilayered  active  and  passive  measures,  within  the  air,  land,  maritime,  and 
space  domains  and  the  information  environment  across  the  range  of  military 
operations  with  an  acceptable  level  of  risk.  (Chambal,  2001,  pp.  Ch  3  25-26) 

This  detailed  definition  shows  the  breadth  and  complexity  of  force  protection  in  a  joint 

environment. 

The  Protection  Joint  Functional  Concept  (Protection  Assessment  Branch,  Joint 
Staff,  2004)  describes  a  construct  for  conducting  force  protection  operations  that  includes 
five  functions:  detect,  assess,  warn,  defend,  and  recover.  The  basic  process  of  this 
construct  is  to  detect  an  attack  either  prior  to  or  during  its  execution  and  then  assess  the 
available  information  in  order  to  make  a  decision  on  how  to  respond  to  the  attack.  The 
decision  on  how  to  respond  will  result  in  warnings  and  or  taskings  to  various  units  to  take 
the  appropriate  defensive  action  to  repel  or  mitigate  the  effects  of  the  attack.  Once  the 
attack  is  over,  it  may  be  necessary  to  conduct  recovery  operations  to  restore  military 
capability. 

2.2.  Fundamentals  of  Architecture  and  the  DoDAF 

According  to  the  Protection  Joint  Functional  Concept,  “current  (force)  protection 
efforts  are  characterized  by  channelized  and  sometimes  conflicting  efforts  among  the 
DoD  agencies,  combatant  commands,  and  Services”  (Protection  Assessment  Branch, 

Joint  Staff,  2004).  In  order  to  achieve  a  unified  and  cooperative  effort  in  the  procurement 
of  physical  security  equipment  for  the  purpose  of  force  protection,  the  Security 
Equipment  Integration  Working  Group  (SEIWG)  is  trying  to  establish  a  DoDAF- 
compliant  architecture  in  the  Joint  Force  Protection  Advanced  Security  System  (JFPASS) 
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initiative  that  describes  joint  force  protection  as  a  guide  for  acquisition  efforts  across  the 
services. 

The  DoDAF  encompasses  29  architectural  products  or  “views”  that  can  be  used  to 
describe  a  myriad  of  complex  technical,  physical,  and  conceptual  systems  (Department  of 
Defense,  2007).  The  views  are  divided  into  four  main  categories:  All  Views  (AV), 
Operational  Views  (OV),  Systems  and  Services  Views  (SV),  and  Technical  Views  (TV). 
Each  view  is  tailored  to  provide  information  on  different  aspects  of  the  system  with  the 
different  categories  focusing  on  broad  areas  of  the  system.  The  two  types  of  AVs  provide 
general  overview  and  background  information  as  well  as  define  the  terms  used  in  the 
architecture.  The  OVs  describe  the  operational  functions  and  structure  of  the  system. 

The  SVs  detail  the  specific  sub-system  and  components  that  make  up  the  system  and 
describes  their  interfaces  and  information  exchanges.  The  two  types  of  TVs  focus  on  the 
current  technical  standards  and  how  the  technical  standards  are  forecast  to  change  over 
time  (Department  of  Defense,  2007).  Table  1  lists  all  of  the  DoDAF  views  and  their 
titles. 

A  set  of  views  describing  a  single  system  is  called  an  architecture  (Department  of 
Defense,  2007).  Architecture  is  a  useful  tool  for  the  management  of  large  organizations 
and  in  particular  joint  missions  that  are  employing  sophisticated  systems  and  technology. 
It  is  also  extensively  used  in  systems  engineering  to  describe  technical  systems  under 
development.  The  use  of  architecture  to  describe  joint  force  protection  provides  a 
structured  and  repeatable  method  for  the  analysis  of  investment  alternatives  for  creating 
new  physical  security  equipment  and  technology  (Department  of  Defense,  2007). 
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The  utility  of  systems  architecture  is  so  important  to  the  DoD  that  the  creation  of 
architectural  views  is  mandated  by  the  Chairman  of  the  Joint  Chiefs  of  Staff.  The  Joint 
Capabilities  Integration  and  Development  System  (JCIDS)  prescribes  a  gated  system 
development  process  that  requires  approval  for  movement  from  one  phase  of 
development  to  the  next.  At  each  transition  point  from  one  phase  to  the  next,  referred  to 
as  a  milestone,  specific  DoDAF  views  are  required  to  be  submitted  for  review  by  a 
milestone  decision  authority  (Chairman  of  the  Joint  Chiefs  of  Staff,  2007).  DoDAF 
volume  II  (2007)  provides  further  guidance  on  choosing  additional  views  for 
development  depending  on  the  purpose  of  the  architecture.  There  are  17  potential  uses 
for  architecture  listed  in  the  DoDAF,  and  views  are  suggested  for  each  use  of  architecture 
to  be  considered  for  development  (Department  of  Defense,  2007).  However,  with  as 
many  as  20  out  of  22  views  to  consider  for  a  particular  use,  this  does  not  significantly 
narrow  the  area  of  consideration  for  the  architect.  Additionally,  there  may  be  many  more 
uses  for  architecture  than  the  17  listed.  Table  2  details  the  uses  and  views  suggested  for 
each.  At  this  point,  the  DoDAF  does  not  distinguish  between  views  with  the  same 
number;  for  instance,  there  is  no  differentiation  between  the  SV-4a  and  SV-4b. 
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Table  1.  The  DoDAF  Views  (Department  of  Defense,  2007) 


View 

Title 

AV-1 

Overview  and  Summary  Information 

AV-2 

Integrated  Dictionary 

OV-1 

High-Level  Operational  Concept  Graphic 

OV-2 

Operational  Node  Connectivity  Description 

OV-3 

Operational  Information  Exchange  Matrix 

OV-4 

Organizational  Relationships  Chart 

OV-5 

Operational  Activity  Model 

OV-6a 

Operational  Rules  Model 

OV-6b 

Operational  State  Transition  Description 

OV-6c 

Operational  Event-Trace  Description 

OV-7 

Logical  Data  Model 

SV-1 

Systems  Interface  Description  Services  Interface  Description 

SV-2 

Systems  Communications  Description  Services  Communications  Description 

SV-3 

Systems-Systems  Matrix  Services-Systems  Matrix  Services-Services  Matrix 

SV-4a 

Systems  Functionality  Description 

SV-4b 

Services  Functionality  Description 

SV-5a 

Operational  Activity  to  Systems  Function  Traceability  Matrix 

SV-5b 

Operational  Activity  to  Systems  Traceability  Matrix 

SV-5c 

Operational  Activity  to  Services  Traceability  Matrix 

SV-6 

Systems  Data  Exchange  Matrix  Services  Data  Exchange  Matrix 

SV-7 

Systems  Performance  Parameters  Matrix  Services  Performance  Parameters  Matrix 

SV-8 

Systems  Evolution  Description  Services  Evolution  Description 

SV-9 

Systems  Technology  Forecast  Services  Technology  Forecast 

SV-lOa 

Systems  Rules  Model  Services  Rules  Model 

SV-1  Ob 

Systems  State  Transition  Description  Services  State  Transition  Description 

SV-lOc 

Systems  Event-Trace  Description  Services  Event-Trace  Description 

SV-11 

Physical  Schema 

TV-1 

Technical  Standards  Profile 

TV-2 

Technical  Standards  Forecast 
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Table  2.  Architecture  Products  by  Uses  (Department  of  Defense,  2007) 


Tech 


All 


Stds 


Uses  of  Architecture 


View 


Operational  View 


System  and  Services  View 


View 


D 

E 

D 

E 

E 

EE 

E 

E 

E 

E 

E 

EE 

E 

E 

E 

E 

EE 

ED 

E 

E 

Analysis  &  Assessment 

Capabilities 

Gaps/Shortfalls 

B 

B 

□ 

□ 

B 

B 

□ 

□ 

□ 

□ 

Mission  Effects  &  Outcomes, 
Operational  Task  Performance 

X 

X 

X 

X 

X 

X 

X 

1 

X 

X 

1 

X 

X 

1 

1 

X 

X 

Trade-Offs 

m 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

Functional  Solutions 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

Operations 

Process  Re-engineering 

H 

B 

B 

B 

B 

B 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Personnel  &  Organizational 
Design 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Doctrine 

De  velopment/V  alidation 

X 

X 

X 

X 

X 

1 

X 

X 

1 

1 

1 

Operational  Planning 
(CONOPS  and  TTPs) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Systems/Services 

Communications 

B 

B 

B 

B 

B 

□ 

□ 

B 

B 

B 

Interoperability  and 
Supportability 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

1 

X 

X 

X 

1 

X 

X 

X 

Evolution/Dependencies 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

Materiel  Solutions  Design  & 
Development 

X 

X 

1 

X 

X 

1 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Facilities  Packaging 

B 

B 

B 

B 

B 

B 

B 

B 

0 

B 

B 

Performance 

□ 

□ 

□ 

B 

B 

□ 

□ 

□ 

B 

B 

B 

□ 

□ 

Socialization/Awareness/Discovery 

Training 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

Leadership  Development 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

Metadata  (for  federation) 

B 

B 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

B 

B 

Guidance  on  the  build  sequence  of  architectural  views  is  provided  in  the  DoDAF 
Version  1  Deskbook  (2003).  Although  DoDAF  version  1.5  does  not  include  an  updated 
deskbook,  the  version  1  deskbook  still  contains  pertinent  information  for  developers  of 
architecture.  The  suggested  “build  sequence,”  reproduced  in  Figure  1,  is  constructed  to 
take  advantage  of  the  relationship  between  products  and  entities  within  products.  The 
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“build  sequence”  takes  advantage  of  information  contained  in  multiple  views  to  reduce 
the  duplication  of  work.  It  also  identifies  steps  to  be  taken  along  the  way  to  a  particular 
view  to  ensure  completeness  of  that  view.  The  process  should  be  iterative,  but  building 
views  in  a  logical  sequence  will  help  insure  data  integrity  (Department  of  Defense,  2003) 


Figure  1.  Suggested  “Build  Sequence”  (Department  of  Defense,  2003) 
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The  DoDAF  suggested  “build  sequence”  is  not  a  sequence  in  the  true  sense  of  the 
word,  since  it  does  not  present  the  views  in  a  single  order  and  there  is  not  a  single  order 
suggested  by  the  diagram.  In  effect  the  “build  sequence”  is  an  activity  on  node  network 
diagram  similar  to  that  used  in  project  management  with  each  view  representing  an 
activity  (Merideth  &  Mantel,  2006).  As  a  result,  several  heuristic  methods  from  the 
project  management  field  can  be  used  to  solve  for  an  actual  build  sequence  based  on  a 
project’s  constraints,  such  as  the  number  of  views  that  can  be  generated  at  a  given  time. 
The  heuristics  work  be  applying  a  given  criteria  to  the  selection  of  an  activity  to 
accomplish  from  the  list  of  activities  available.  An  activity  is  available  when  all  of  its 
prerequisites  are  complete.  The  heuristics  apply  criteria  that  range  from  selecting  the 
activity  with  the  shortest  duration  to  selecting  the  activity  that  adds  the  most  value  to  the 
project  (Merideth  &  Mantel,  2006). 

2.3.  Evaluation  of  Architecture  and  DoDAF 

System  engineering  adds  value  to  the  system  design  process.  Honour  (2004) 
suggests  that  there  is  some  correlation  between  the  level  of  systems  engineering  effort 
and  both  the  project  development  quality  and  the  relative  success  of  the  project.  Honour 
(2004)  looks  at  the  system  engineering  effort  as  a  whole,  to  include  system-architecting 
efforts,  and  uses  only  a  qualitative  evaluation  of  system  engineering  made  by  project 
participants.  However,  this  subjective  evaluation  of  systems  engineering  does  not 
specifically  address  the  quality  of  the  architecture  or  the  value  it  adds  to  the  overall 
project. 

A  review  of  existing  system  architecture  literature  reveals  that  techniques  and 
methodologies  for  specifically  evaluating  architecture  are  limited  and  mostly  focus  on  a 
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single  aspect  or  single  type  of  system.  For  example,  the  i-Score  methodology  examines 
system  interoperability  by  analyzing  the  interoperability  of  system  pairs  within  a 
sequence  of  activities  (Ford,  Colombi,  Graham,  &  Jacques,  2007).  The  score  derived 
from  this  methodology  represents  the  quality  of  a  single  aspect  of  architecture  and  does 
not  indicate  the  overall  quality  or  value  of  that  architecture.  The  Architecture  Tradeoff 
Analysis  Method  is  designed  for  analyzing  the  tradeoffs  in  architecture  decisions  for 
software  systems  (Kazman,  Klein,  &  Clements,  2000).  The  use  of  executable  models  and 
simulation  can  be  used  to  test  and  validate  the  design  of  well  developed  systems, 
particularly  in  the  areas  of  the  system’s  process  logic  and  software  (Levis,  Shin,  & 
Bienvenu,  2000;  Levis,  Wagenhals,  Shin,  &  Kim,  2000). 

The  Institute  for  Enterprise  Architecture  Development  has  created  a  scorecard 
establishing  criteria  to  guide  the  review  of  enterprise  architecture.  The  Enterprise 
Architecture  Scorecard  (2007)  evaluates  the  enterprise  areas  of  organization,  information, 
information-systems,  and  technology-infrastructure  at  six  levels  of  abstraction.  These  six 
levels  of  abstraction  represent  the  six  typical  areas  of  concern  for  architecture:  contextual, 
environmental,  conceptual,  logical,  physical,  and  transformational.  This  evaluation 
provides  24  distinct  areas  for  a  reviewer  to  look  at  when  evaluating  an  architecture  as 
shown  in  Figure  2.  The  evaluation  is  based  on  a  subjective  assignment  of  a  score  to  a 
series  of  broad  questions  (Institute  For  Enterprise  Architecture  Developments,  2007). 

This  type  of  evaluation  may  give  an  indication  as  to  whether  the  architecture  was 
developed  thoroughly  enough  as  to  sufficiently  address  all  24  areas  being  evaluated,  but 
no  link  is  made  from  the  areas  of  evaluation  to  overall  quality  of  the  products  or  of  their 
value  to  a  design  program. 
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Why? 
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How? 

Logical  Representation 

Logical  Level 

With  what? 

Solution 

Representation 

Physical  Level 

When? 

Enterprise  Impact 

Transformational  Level 

Business 

Information 

Areas  of  Evaluation 

Information  - 
Systems 

Technology  - 
Infrastructure 

Figure  2.  The  Enterprise  Architecture  Scorecard  (Institute  For  Enterprise 
Architecture  Developments,  2007) 


2.4.  Value-Focused  Thinking  (VFT) 

“Values  are  principles  used  for  evaluation.  We  use  them  to  evaluate  the 
actual  or  potential  consequences  of  action  and  inaction,  of  proposed 
alternatives,  and  of  decisions”  (Keeney,  1992,  p.  6). 

Value-Focused  Thinking  is  a  decision  analysis  tool  that  seeks  to  improve 

decision-making  by  quantifying  the  values  of  a  decision-maker,  generating  alternatives 

based  on  those  values,  and  comparing  alternatives  against  those  values.  A  decision  by 

definition  has  multiple  alternatives  that  will  have  differing  consequences  in  terms  of 

outcome  and  resource  consumption  (Kirkwood,  1997).  Many  times,  decisions  are  made 

by  focusing  on  obvious  alternatives  and  selecting  the  best  among  them.  This  type  of 

alternative-focused  decision-making  can  be  done  quickly,  but  can  inadvertently  limit 

analysis  to  a  small  number  of  undesirable  alternatives.  One  of  the  benefits  of  value- 

focused  thinking  is  that  it  is  unconstrained  by  alternatives  and  enables  the  decision-maker 
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to  articulate  an  ideal  consequence  and  then  go  about  looking  for  or  creating  an  alternative 
that  best  matches  the  ideal  (Keeney,  1994). 

2.4.1.  Benefits  of  Value-Focused  Thinking 

The  major  difference  between  a  typical  decision-making  process  and  value- 
focused  thinking  is  a  thorough  value  assessment.  In  many  situations,  the  relevant  values 
to  a  decision  are  known  intuitively  and  a  value  assessment  merely  articulates  those 
values.  In  other  more  complex  decision  problems,  not  all  of  the  relevant  values  may  be 
as  obvious  or  other  stakeholders  may  obscure  values.  A  detailed  value  assessment  can 
either  discover  or  uncover  these  unknown  or  obscured  values  (Keeney,  1994).  This  is 
important  since  values  represent  the  reason  for  interest  in  any  decision  problem  and,  as 
such,  values  can  provide  useful  insights  to  all  aspects  of  the  decision-making  process 
(Keeney,  1994). 

For  many  decision  situations,  there  are  multiple  stakeholders.  For  instance,  each 
of  the  four  service  branches  of  the  DoD  represent  a  stakeholder  in  joint  force  protection, 
each  with  their  own  set  of  values  and  objectives.  These  differing  value  sets  between 
stakeholders  often  lead  to  disagreement  over  the  acceptability  of  various  alternatives.  If 
the  discussion  about  the  decision  situation  focuses  solely  on  the  identified  alternatives, 
conflict  will  likely  arise  as  the  discussion  turns  into  an  argument  over  alternatives 
(Keeney,  1992).  If  the  discussion  is  about  the  values  of  various  stakeholders  though,  the 
underlying  reason  for  disagreement  can  be  uncovered;  additionally,  it  is  likely  that  the 
stakeholders  share  many  of  the  same  values.  Understanding  the  similarities  and 
differences  in  values  between  stakeholders  can  lead  to  new  alternatives  that  better  meet 
the  objectives  of  all  the  stakeholders.  Even  just  the  effort  of  identifying  the  values  of  a 
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stakeholder  and  incorporating  them  into  the  decision  process  encourages  stakeholder 
support  for  the  decision  (Keeney,  1992;  Kirkwood,  1997). 

The  process  of  conducting  a  value  assessment  for  strategic  level  values  is  also 
extremely  beneficial  as  these  values  are  stable  over  time  and  therefore  can  be  reused  to 
analyze  new  decision  problems  and  guide  efforts  for  achieving  strategic  objectives 
(Kirkwood,  1997;  Keeney,  1992).  The  strategic  values  of  joint  force  protection  should 
not  change  in  the  near  future  and  any  changes  that  occur  should  be  minor;  therefore,  the 
value  hierarchy  for  joint  force  protection  can  be  a  long-term  source  of  agreement  among 
stakeholders. 

2.4.2.  Previous  Application  of  VFT  in  DoD 

The  VFT  methodology  has  been  used  in  a  variety  of  applications  within  the  DoD. 
A  brief  review  of  these  applications  shows  the  versatility  of  VFT  due  to  the  universal 
applicability  of  values  to  decision  problems.  Knighton  (1998)  used  VFT  to  explore  the 
problem  of  course  scheduling  at  the  Air  Force  Institute  of  Technology.  Using  the 
Institute’s  objectives,  instructor  and  student  preferences,  and  facility  constraints,  an 
automated  course-scheduling  tool  was  created  to  solve  a  complex  scheduling  problem. 
This  research  incorporated  the  ability  of  VFT  to  balance  objectives  of  multiple 
stakeholders  with  physical  constraints  to  create  the  best  alternative  course  schedule 
(Knighton,  1998). 

Shoviak  (2001)  applied  VFT  to  the  selection  of  a  solid  waste  management  site  for 
an  Air  Station  in  Alaska.  In  his  research  into  this  problem,  Shoviak  (2001)  used  the  10- 
step  process  for  the  application  of  VFT  that  was  the  basis  for  the  VFT  work  done  in  this 
research.  Jurk  (2002)  applied  the  same  methodology  to  the  selection  of  force  protection 
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initiatives  for  further  development.  Keeter  (2005)  applied  VFT  to  effects-based 
operations  demonstrating  the  versatility  of  the  process  to  be  applied  to  multiple  decision 
contexts  including  day-to-day  operations. 

2.5.  Measuring  Value 

An  effectiveness  measure  seeks  to  provide  information  on  the  performance  of  a 
system  or  progress  towards  a  desired  end  state  (Bullock,  2006).  Measuring  performance 
is  critical  to  the  management  of  a  process  or  organization.  The  simple  fact  that  a 
particular  metric  is  measured  is  often  motivation  enough  for  an  organization  to  improve 
performance  with  regard  to  that  metric.  However,  many  organizations  only  measure 
financial  metrics  even  though  they  profess  the  importance  of  performance  in  non- 
financial  areas  (Kaplan  &  Norton,  1996).  This  focus  on  financial  indicators  is  criticized 
because  it  often  encourages  efforts  to  increase  short-term  financial  results  at  the  expense 
of  long-term  value  creation  (Porter,  1992).  The  challenge  is  how  to  measure  the  long¬ 
term  creation  of  intangibles  that  often  compete  with  the  short-term  financial  rewards.  For 
example,  the  intangibles  of  knowledge  and  expertise  often  compete  with  the  financial 
benefits  of  outsourcing  (Kaplan  &  Norton,  1996). 

Several  management  tools  are  available  to  tackle  the  challenge  of  measuring 
performance  with  respect  to  a  strategic  vision  or  mission  statement.  For  instance,  the 
Balanced  Scorecard  approach  as  presented  by  Kaplan  and  Norton  (1996)  provides  a 
“framework  that  translates  a  company’s  vision  and  strategy  into. .  .performance 
measures.”  The  Balanced  Scorecard  approach  focuses  on  performance  from  four 
perspectives:  financial,  customer,  internal-business-process,  and  learning  and  growth. 

The  idea  is  to  link  the  objectives  from  a  mission  statement  to  the  measures  in  each  of  the 
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perspectives  to  ensure  a  balanced  approach  to  performance  measurement  (Kaplan  & 
Norton,  1996).  This  is  accomplished  through  a  10-step  process  that  incorporates  an 
organization’s  vision,  mission,  and  strategy  with  input  from  senior  executives  to  establish 
strategic  objectives  and  identify  measures  for  each  objective  (Kaplan  &  Norton,  1996). 
The  Balanced  Scorecard  process  of  establishing  objectives  from  strategic  guidance  and 
input  of  senior  executives  resembles  the  VFT  process  of  eliciting  an  objective  hierarchy 
from  decision-makers.  The  advantages  of  VFT  over  balanced  scorecard  are  the  added 
steps  of  weighting  objectives  to  guide  the  balancing  of  competing  objectives  and  creating 
single  dimensional  value  functions  that  facilitates  the  comparison  between  alternatives. 

The  “cause-and-effect”  matrix  is  another  management  tool  that  analyzes  how 
process  steps  contribute  to  customer  requirements  (Tague,  2005).  Table  3  shows  an 
example  “cause-and-effect”  matrix.  The  customer  requirements,  or  “output  variables,” 
are  weighted  in  terms  of  importance  to  the  customer  and  listed  across  the  top  of  the 
matrix.  The  process  steps  or  “input  variables”  are  listed  down  the  side  of  the  matrix.  The 
influence  of  each  input  variable  is  rated  against  each  output  variable  on  a  scale  of  1  to  3, 
where  1  is  little  influence  and  3  is  highly  influential;  this  is  the  first  number  in  each 
input/output  intersection  of  Table  3.  The  influence  ratings  of  each  input  variable  are  then 
multiplied  by  the  importance  weight  of  the  corresponding  output  variables;  the  resulting 
score  is  the  second  number  in  each  input/output  intersection  of  Table  3.  This  second 
number  is  summed  for  each  input  variable.  The  result  is  a  score  for  each  input  variable 
that  shows  its  relative  influence  on  the  output  variables;  this  information  can  be  used  to 
allocate  resources  for  improving  the  process  being  analyzed  (Tague,  2005). 
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Table  3.  Sample  “Cause-and-Effect”  Matrix 


Outputs 

Output  1 

Output  2 

Output  3 

Total 

Weights 

1 

4 

2 

Input  1 

Influence 

Score 

Influence 

Score 

Influence 

Score 

13 

3 

3 

1 

4 

3 

6 

Input  2 

Influence 

Score 

Influence 

Score 

Influence 

Score 

17 

1 

1 

3 

12 

2 

4 

Input  3 

Influence 

Score 

Influence 

Score 

Influence 

Score 

14 

2 

2 

2 

8 

2 

4 

Research  has  also  been  done  to  specifically  assess  the  value  of  an  activity  or 
resource  in  the  context  of  achieving  a  set  of  objectives.  The  Research  and  Development 
(RAND)  Corporation  was  contracted  by  the  U.S.  Army  to  assess  the  value  of  Army 
International  Activities  in  accomplishing  DoD  objectives  (RAND  Arroyo  Center,  2006). 
Additionally,  Jones  (2006)  developed  a  methodology  for  military  planners  to  assess  the 
value  of  resources  for  accomplishing  objectives  in  effects-based  operations.  The  RAND 
Corporation  and  Jones  research  efforts  are  discussed  in  more  detail  below. 

2.5.1.  Assessing  the  Value  of  U.S.  Army  International  Activities 

Since  the  Cold  War,  the  DoD  has  developed  a  more  flexible  and  comprehensive 
international  cooperation  strategy  to  deal  with  a  more  complex  strategic  environment. 
This  has  meant  an  increased  profile  and  focus  for  the  Army  International  Activities 
Program.  Because  of  the  increased  focus,  a  need  has  arisen  to  assess  the  contribution  of 
Army  international  activities  to  higher-level  DoD  objectives  and  improve  decision¬ 
making  on  resource  allocation.  To  fill  this  gap,  the  Army  sponsored  a  research  project  by 
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the  RAND  Corporation  to  assess  the  value  of  U.S.  Army  International  Activities  (RAND 
Arroyo  Center,  2006). 

2.5.1. 1.  Overview  of  research 

The  major  research  problem  the  RAND  Corporation  wanted  to  address  was  to  link 
Army  international  activities  to  the  accomplishment  of  strategic  level  objectives  and 
measure  the  extent  to  which  individual  activities  contribute  to  achieving  those  objectives. 
In  order  to  accomplish  this,  several  issues  needed  to  be  addressed  such  as  multiple 
stakeholders  with  various  responsibilities,  multiple  objectives  of  different  types  and  time 
horizons,  and  a  diverse  set  of  programs  that  made  side-by-side  comparisons  difficult 
(RAND  Arroyo  Center,  2006). 

The  objectives  for  measuring  the  Army  International  Activities  Program  were 
derived  in  a  multi-step  process  that  began  with  the  development  of  a  set  of  objectives  for 
security  cooperation.  Then  a  set  of  “ends”  or  lower-level  objectives  for  the  Army 
International  Activities  Program  were  established.  Since  this  hierarchy  of  objectives  was 
derived  from  government  policy  and  directives  such  as  the  National  Security  Strategy, 
the  Quadrennial  Defense  Review,  and  the  Security  Cooperation  Guidance,  it  also  reflects 
a  reasonable  set  of  objectives  that  any  state  might  pursue  through  security  cooperation 
(RAND  Arroyo  Center,  2006). 

Next,  the  various  Army  international  activities  were  grouped  into  categories  or 
“ways”  such  as  education  and  training,  personnel  exchanges,  and  exercises.  Grouping 
the  large  and  diverse  number  of  activities  into  a  smaller  number  of  “ways”  was  done  to 
make  an  assessment  more  manageable.  The  results  of  the  objectives  hierarchy  with  its 
lowest-level  “ends”  and  the  grouping  of  activities  into  “ways”  is  the  “ends-by-ways” 
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matrix  shown  in  Figure  3.  This  matrix  was  used  to  identify  measures  of  effectiveness 
that  evaluated  the  contribution  of  each  “way”  to  each  “end.”  This  process  provided 
information  on  the  magnitude  of  the  contribution  of  each  “way”  to  each  “end”  and  how 
some  “ways”  can  contribute  to  multiple  “ends”  (RAND  Arroyo  Center,  2006). 


i  The  Ends  (from  AIAP,  TAP,  DPG,  QDR,  and  NSS )  i 
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Figure  3.  "Ends-by-Ways  Matrix"  for  Army  International  Activities  (RAND 

Arroyo  Center,  2006) 


2.5.I.2.  Similarity  to  VFT 

The  RAND  Corporation’s  use  of  policy  and  strategy  documents  to  ascertain  a  list 
of  eight  objectives  for  Army  international  activities  mirrors  the  Gold  Standard  VFT 
methodology  that  relies  on  documentation  of  a  decision-maker’s  objectives  or  intent  to 
create  an  objectives  hierarchy  (Burk  &  Parnell,  1997).  The  eight  activity  categories 
represent  alternatives  for  achieving  the  eight  objectives  that  can  be  scored  individually  or 
as  a  set  of  activities.  One  of  the  challenges  of  the  RAND  Corporation’s  research  (2006) 
was  to  establish  causal  linkages  between  activities  and  achieving  objectives  and  from 
there  developing  Measures  of  Effectiveness  that  show  how  well  different  activities 
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accomplish  objectives  (RAND  Arroyo  Center,  2006).  These  Measures  of  Effectiveness 
equate  to  evaluation  measures  in  the  10-step  VFT  process.  The  missing  piece  is  the 
single  dimensional  value  function  that  translates  evaluation  measures  into  a 
dimensionless  value  scale  for  comparison  (Shoviak,  2001). 

2.5.2.  Resource  Value  in  Effects-Based  Operations 

The  initial  step  of  effects-based  operations  is  to  clearly  define  a  desired  end-state 
and  “translate  the  desired  end-state  into  fundamental  campaign  objectives”  (Jones,  2006). 
It  is  the  job  of  the  military  planners  to  utilize  available  resources  to  accomplish  those 
objectives.  Unfortunately,  campaign  objectives  can  compete  for  resources  or  directly 
conflict  in  terms  of  level  of  accomplishment;  for  example,  the  destruction  of  key  enemy 
command  and  control  nodes  within  a  city  may  directly  conflict  with  the  desire  to  limit 
collateral  damage.  Jones  (2006)  presents  a  methodology  for  examining  the  value  of 
resources  in  terms  of  accomplishing  objectives  and  the  interconnections  between 
objectives  for  the  purpose  of  aiding  military  planners  in  allocating  resources. 

The  methodology  presented  by  Jones  (2006)  assumes  that  national  leadership  and 
military  commanders  have  provided  an  objectives  hierarchy  and  that  single  actions  can 
affect  the  achievement  of  multiple  objectives  both  positively  and  negatively.  This  creates 
a  situation  similar  to  the  RAND  Corporation’s  study  (2006),  in  which  each  action 
represents  an  alternative  that  can  be  scored  against  a  set  of  objectives  either  individually 
or  as  a  set  of  actions.  The  degree  to  which  an  objective  is  met  is  converted  to  a 
dimensionless  value  scale  ranging  from  zero  to  unity  using  a  value  function  so  that 
objectives  can  be  compared  on  equivalent  scales.  In  this  manner,  the  progress  of  the 
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campaign  can  be  measured  in  terms  of  its  relative  progress  toward  the  desired  end-state 
that  would  be  represented  by  a  perfect  score  for  all  objectives  (Jones,  2006). 

Additionally,  the  value  of  each  resource  can  be  measured  in  “the  context  of  the 
degree  of  attainment  of  campaign  objectives”  (Jones,  2006).  This  can  be  done  for  a 
resource’s  value  in  attaining  a  single  objective  but  is  more  useful  in  the  context  of  all 
stated  objectives  as  some  objectives  may  be  conflicting.  Evaluating  resources  against  all 
objectives  requires  a  weighted  value  hierarchy  from  which  a  multiple  objective  value 
function  can  be  generated  to  calculate  a  resource’s  contribution  to  the  overall  campaign 
objective  (Jones,  2006). 
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Chapter  3.  Methodology 


As  discussed  in  Chapter  1,  the  Department  of  Defense  Architectural  Framework 
(DoDAF)  describes  29  standardized  architectural  views  for  use  in  describing  complex 
systems.  A  primary  reason  for  creating  an  architecture  is  to  support  decision-making  and 
communication  in  the  Department  of  Defense  (DoD)  (Department  of  Defense,  2007). 

The  development  of  particular  architecture  views  is  required  for  milestone  decision 
points  under  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  process 
(Chairman  of  the  Joint  Chiefs  of  Staff,  2007).  However,  development  of  each  view  is 
both  time  and  resource  intensive;  requiring  input  from  subject  matter  experts  and  system 
engineers  with  specialized  training.  The  creation  of  all  29  views  is  not  practical  and 
likely  not  optimal  to  system  development.  Some  guidance  exists  to  help  the  architect 
select  views,  but  this  is  limited  and  provides  only  a  narrowed  field  of  views  from  which 
to  select  (Department  of  Defense,  2007). 

The  methodology  presented  here  is  an  extension  of  the  Value-Driven  Enterprise 
Architecture  (VDEA)  score  and  is  complemented  by  the  work  of  Cotton  and  Haase 
(2009)  and  Mills  (2009)  in  the  application  of  Value-Focused  Thinking  (VFT)  to 
architectural  analysis.  The  purpose  of  this  methodology  is  to  provide  the  manager  of  an 
architectural  design  effort  a  tool  for  view  selection  as  well  as  provide  insight  into  the 
importance  of  individual  views  and  groupings  of  views.  The  application  of  the 
methodology  presented  here  is  specific  to  the  problem  of  Joint  Force  Protection  but  the 
methodology  is  applicable  to  other  problems  as  an  add-on  to  VDEA.  Further  application 
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of  this  methodology  may  provide  further  insight  into  the  value  of  individual  and 
groupings  of  views  as  they  apply  to  different  types  of  problems. 

3.1.  Data  Collection 

Data  collection  for  this  research  followed  the  initial  steps  of  the  10-step  VFT 
methodology  first  developed  by  Chambal  (2001).  This  methodology  was  selected 
because  of  its  simplicity,  which  aids  in  communication  with  the  decision-maker.  The 
extensive  application  of  the  10-step  VFT  methodology  in  the  DoD  includes  utility 
privatization  (Braziel,  2004),  selection  of  force  protection  initiatives  (Jurk,  2002),  and 
strategic  airlift  (Tharaldson,  2006).  The  multiple  successful  applications  of  the  10-step 
process  provide  validity  to  the  tool  and  its  extension  to  a  Joint  Force  Protection  Advanced 
Security  System  (JFPASS)  Architecture.  This  application  of  the  10-step  process  was 
iterative  with  frequent  feedback  loops  to  previous  steps  to  make  changes  and  revalidate 
results. 

The  first  step  of  the  10-step  process  was  to  identify  the  problem.  It  was  important 
to  ensure  that  the  problem  and  its  scope  was  understood  by  all  parties  to  the  decision  as 
this  defined  the  boundary  of  the  JFPASS  architecture  decision  context  and  greatly 
influenced  the  resulting  value  hierarchy.  The  second  step  was  to  develop  the  value 
hierarchy  that  was  applicable  to  the  decision  context  and  in  accordance  with  the  values  of 
the  Security  Equipment  Integration  Working  Group  (SEIWG)  chairman  and  subject 
matter  experts  in  force  protection  and  architecture  (Chambal,  2001).  The  values  at  the 
lowest  level  of  the  value  hierarchy  are  referred  to  as  the  last  or  lowest- tier  values. 

In  step  3  for  each  lowest-tier  value,  an  evaluation  measure  was  developed. 
Evaluation  measures  were  either  direct  measures  or  proxy  measures.  A  direct  evaluation 
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measure  was  preferred  as  it  directly  measures  the  attainment  of  a  value;  however,  in  some 
instances  this  is  not  practical  or  possible  (Chambal,  2001).  For  instance,  the  national 
economic  health  is  difficult  to  measure  directly;  therefore,  the  Gross  Domestic  Product  is 
used  as  a  proxy  in  most  cases.  Evaluation  measures  also  have  either  natural  or 
constructed  scales.  A  natural  scale  is  one  that  is  commonly  used  for  that  type  of 
measurement;  for  instance,  the  speed  of  an  automobile  is  typically  measured  in  miles  per 
hour.  In  many  cases,  a  natural  scale  was  not  available  and  a  scale  needed  to  be 
constructed  for  this  research.  A  constructed  scale  would  be  the  construction  quality  of  an 
automobile  rated  as  low,  medium,  or  high.  In  general,  a  natural  scale  is  preferred  over  a 
constructed  one  as  a  natural  scale  is  already  widely  used  and  understood  (Chambal, 

2001). 

Once  the  evaluation  measures  were  determined,  step  4  was  to  create  a  single 
dimensional  value  function  for  each  measure.  A  single  dimensional  value  function 
transposes  an  evaluation  measure  from  the  scale  in  which  it  is  measured  to  a 
dimensionless  scale  of  value  ranging  from  zero  to  unity.  The  use  of  a  common  value 
scale  for  all  evaluation  measures  allows  for  the  summation  of  the  measures  to  obtain  a 
total  score  (Chambal,  2001). 

The  last  step  to  complete  the  hierarchy,  step  5,  was  to  determine  the  relative 
weights  for  the  objectives  at  each  level.  The  weighting  of  objectives  accounts  for  the 
extent  to  which  lower-level  objectives  contribute  to  higher-level  objectives  and  the 
relative  importance  of  objectives  to  the  decision-maker  (Chambal,  2001).  For  the 
purposes  of  this  research,  the  completed  objectives  hierarchy,  and  specifically  the  global 
measure  weights,  is  all  that  is  required  from  this  process.  This  research  uses  the 
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weighting  as  a  proxy  for  importance  to  evaluate  architecture  views  and  answer  the 
research  questions  from  Chapter  1. 

3.2.  Ways  to  Means  Analysis 

At  this  point,  this  thesis  diverts  from  the  10-step  VFT  to  tools  similar  to  those 
used  by  the  RAND  Corporation  (RAND  Arroyo  Center,  2006)  and  Jones  (2006);  it 
resembles  the  “cause-and-effect”  matrix  and  associated  methodology  described  by  Tague 
(2005).  However,  this  application  differs  in  several  significant  ways.  The  effects  or  ends 
being  examined  are  the  objectives  of  joint  force  protection  architecture  as  elicited  from 
multiple  stakeholders  using  the  VFT  process.  The  resources,  causes,  or  ways  being 
examined  are  the  29  architectural  views  described  by  the  Department  of  Defense 
Architecture  Framework  (DoDAF).  The  process  used  in  this  research  is  detailed  below. 

3.2.1.  Create  Matrix  and  Identify  Relationships 

The  first  step  is  to  create  a  matrix  with  the  evaluation  measures  from  the  hierarchy 
on  one  side  and  the  DoDAF  views  across  the  top.  This  matrix  is  the  basis  for  all  further 
analysis.  The  second  step  is  to  identify  the  views  that  can  contribute  to,  or  fulfill  on  their 
own,  a  given  evaluation  measure.  At  this  point,  it  is  assumed  that  any  view  if  done 
correctly  will  not  detract  from  any  evaluation  measure;  it  is  also  assumed  that  any  view 
that  is  created  will  be  done  so  to  a  satisfactory  standard.  Some  evaluation  measures  will 
be  related  to  every  view;  these  are  the  evaluation  measures  that  relate  to  the  quality  of  the 
created  views.  These  measures  should  be  noted  as  such,  as  they  are  non-discriminating 
between  views  and  can  be  left  out  of  the  analysis  since  they  offer  no  insight  into  the 
importance  of  an  individual  view. 
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3.2.2.  Describe  Relationships 

The  third  step  is  to  describe  the  strength  of  relationship  between  each  evaluation 
measure  and  each  view.  The  strength  of  relationship  will  be  described  on  a  scale  from 
zero  to  one,  with  zero  representing  no  relationship  and  one  representing  an  exclusive 
relationship  between  a  particular  view  and  a  particular  measure.  When  more  than  one 
view  is  associated  with  a  measure,  the  strength  of  the  relationship  of  the  measure  to  each 
associated  view  will  be  assumed  to  be  one  over  the  number  of  views  associated  with  that 
measure.  In  other  words,  all  views  associated  with  a  measure  are  assumed  to  contribute 
equally  to  that  measure.  In  the  case  that  a  view  does  not  contribute  to  a  particular 
measure,  the  field  in  the  matrix  is  left  blank  and  takes  the  value  of  zero  for  calculations. 
3.3.  Analysis 

The  analysis  was  done  in  two  parts  to  answer  the  four  research  questions  from 
Chapter  1.  First,  each  view  was  looked  at  individually  to  determine  its  importance  as  a 
single  view  and  which  views  are  most  important  to  the  JFPASS  architecture.  Second,  the 
views  were  rank-ordered  by  importance  and  a  build  sequence  was  generated  using  the 
DoDAF  recommended  network  diagram. 

3.3.1.  View  Analysis 

The  first  part  of  the  analysis  is  the  simplest  since  it  considers  only  one  view  at  a 
time.  The  numerical  relationship  descriptions  for  each  view  are  multiplied  by  the 
corresponding  measure  weight  and  the  products  are  summed  across  all  the  evaluation 
measures.  This  summation  results  in  a  dimensionless  score  for  each  view  that  can  be 
compared  against  the  score  for  other  views.  The  equation  used  for  this  calculation  is: 

Kx)  =  Z"=1Wiii(*f)  (l) 


31 


where  I(x)  is  the  overall  importance  score  of  the  view,  Wj(Xj)  is  the  importance  of  the 
view  on  the  ith  measure,  A;  is  the  weight  of  the  ith  measure,  n  is  the  total  number  of 
measures,  and  x,-  is  the  strength  of  the  relationship  between  view  and  the  ith  measure. 

Listing  the  views  by  score  identifies  the  most  important  and  the  least  important 
views.  This  answers  the  first  research  question  from  Chapter  1.  Comparing  the  sorted 
list  of  views  to  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS) 
required  views  identifies  the  views  that  are  not  required  but  considered  to  be  of 
importance,  and  the  views  that  are  required  but  considered  to  be  of  minimal  importance. 
This  is  the  answer  to  research  questions  3  and  4  from  Chapter  1 .  The  second  phase  of  the 
analysis  will  consider  all  of  the  views  that  are  important  to  the  architecture.  The 
objective  is  to  find  a  collection  of  views  that  can  meet  all  the  evaluation  measures. 

3.3.2.  Build  Sequence 

The  recommended  build  sequence  for  the  quickest  increase  in  utility  is  generated 
using  the  network  diagram  from  the  DoDAF  Deskbook  (2003).  The  network  diagram 
that  DoDAF  suggests  shows  prerequisites  for  each  view  similar  to  the  network  diagram 
method  that  is  frequently  used  in  project  management  (Merideth  &  Mantel,  2006).  The 
network  diagram  is  simplified  to  include  only  the  views  that  are  found  to  be  important  or 
are  prerequisites  for  a  view  that  is  found  to  be  important.  Starting  with  no  views  having 
been  built,  the  network  diagram  is  used  in  conjunction  with  three  heuristics  from  the 
precedence  diagramming  method.  The  first  heuristic  is  based  on  the  order  of  importance 
and  always  selects  the  most  important  view  from  those  available  at  a  given  point  in  the 
network.  The  objective  of  this  heuristic  is  to  achieve  the  “steepest  ascent”  possible  in  the 
growth  curve  of  the  project  at  a  given  decision  point.  The  second  heuristic  is  the  “most 
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successors”  heuristic,  which  selects  the  view  with  the  most  successors  from  those  that  are 


available.  The  last  heuristic  is  similar  to  the  second  but  only  considers  critical  successors 
and  is  known  as  the  “most  critical  followers”  heuristic  (Merideth  &  Mantel,  2006).  For 
this  application,  not  all  views  found  to  be  important  will  be  considered  critical;  only  the 
top  few  most  important  will  be  used,  with  the  exact  delineation  being  subjective. 

3.4.  Limitations 

The  application  of  this  methodology  requires  a  value  hierarchy  tailored  to  the 
exact  architecture  project  being  evaluated.  As  such,  this  research  is  only  intended  as  a 
demonstration  of  the  methodology.  Further,  full  validation  will  require  additional 
applications  to  a  variety  of  architecture  projects.  With  additional  applications,  trends 
may  also  be  identified  that  may  have  wider  implications  on  architectural  development. 
Views  that  are  repeatedly  found  to  be  important  to  architecture  projects  should  be 
included  in  policy  as  requirements  for  milestone  decisions.  Views  that  are  repeatedly 
found  not  to  be  important  to  architecture  projects  should  be  reviewed  to  analyze  their 
continued  value. 


33 


Chapter  4.  Results  and  Analysis 


This  chapter  covers  the  results  of  data  collection  and  the  analysis  of  those  results. 
The  collection  of  data  was  based  on  the  application  of  the  first  five  steps  of  the  10-step 
Value-Focused  Thinking  (VFT)  methodology  to  the  development  of  a  Value-Drive 
Enterprise  Architecture  (VDEA)  score  to  evaluate  a  Joint  Force  Protection  Advanced 
Security  System  architecture.  The  VFT  process  was  accomplished  iteratively  with 
numerous  updates  and  revalidation  of  previous  steps.  This  yielded  a  weighted  objectives 
hierarchy  and  evaluation  measures.  This  chapter  also  describes  the  creation  of  an 
evaluation  “measures-by- views”  matrix  and  how  that  matrix  was  evaluated  to  determine 
the  importance  of  individual  architecture  views  and  develop  recommendations  for  view 
development. 

4.1.  Develop  Value  Hierarchy  and  Value  Hierarchy  Weights 

The  development  of  a  value  hierarchy  involved  literature  review,  affinity 
diagramming,  and  validation  by  Security  Equipment  Integration  Working  Group 
(SEIWG)  members  and  experts  in  force  protection.  A  review  of  pertinent  architecture 
evaluation  literature  and  force  protection  literature  provided  a  frame  of  reference  to  work 
from  as  well  as  a  list  of  “ilities”  associated  with  architecture  and  force  protection  for  an 
affinity  diagramming  exercise.  The  affinity  diagramming  exercise  considered  over  a 
hundred  concepts  and  used  group  consensus  to  arrange  them  in  categories  based  on  their 
similarity  and  associations  with  one  another.  First,  concept  terms  were  placed  in  groups 
based  on  their  similarity  in  meaning,  then  these  groups  were  clustered  based  on  similarity 
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of  the  overarching  concept  they  were  addressing.  From  this  exercise,  a  draft  value 
hierarchy  was  developed.  This  draft  hierarchy  was  then  presented  to  the  SEIWG 
chairman  and  a  wide  variety  of  subject  matter  experts  to  obtain  feedback  and  validation. 
Some  minor  adjustments  to  the  hierarchy  were  made  based  on  the  feedback.  A  finalized 
value  hierarchy  was  validated  by  the  SEIWG  chairman  and  subject  matter  experts. 

With  the  structure  of  the  value  hierarchy  complete,  the  relative  weighting  was 
done  with  the  SEIWG  chairman  and  a  group  of  subject  matter  experts  in  both  architecture 
and  force  protection.  The  weighting  was  accomplished  by  proceeding  from  the  top  of  the 
hierarchy  in  a  branch-by-tier  fashion  and  assigning  local  weights  to  objectives  relative  to 
the  other  objectives  in  a  given  tier  of  a  given  branch.  Adjustments  to  the  hierarchy  were 
made  and  global  weights  were  calculated  in  real  time  to  show  participants  how  the 
changes  being  made  affected  the  hierarchy  as  a  whole. 

The  resulting  hierarchy  has  two  main  branches  representing  the  quality  of  the 
architecture  and  the  effectiveness  of  the  system  being  described  by  the  architecture,  with 
the  effectiveness  of  the  system  accounting  for  60%  of  the  architecture’s  value.  The 
System  Effectiveness  branch  is  broken  into  the  capability,  maintainability,  and 
interoperability  of  the  system.  Capability  is  the  most  important  system  effectiveness 
value  accounting  for  45%  of  the  value  in  that  branch,  with  Maintainability  and 
Interoperability  splitting  the  remaining  55%.  The  Architecture  Quality  branch  is  broken 
into  four  branches  representing  Accessibility,  Usability,  Modifiability,  and  Accountability. 
Of  these  four  values,  Usability  is  the  most  valued,  Accessibility  and  Accountability  are 
equally  valued,  and  Modifiability  is  the  least  valued  of  the  four.  The  definitions  of  all  the 
System  Effectiveness  values  can  be  found  in  Table  4,  while  the  definitions  of  the 
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Architecture  Quality  branch  can  be  found  in  Table  5.  These  tables  are  structured  to 
demonstrate  the  organization  of  the  hierarchy.  For  a  more  detailed  account  of 
Architecture  Quality  values  see  Cotton  and  Haase  (2009).  For  details  on  System 
Effectiveness  values  see  Mills  (2009). 

A  comparison  of  the  weights  of  each  of  the  lowest-tier  values  shows  that 
Purposefulness  is  the  most  important  value  trailed  by  Communication.  The  System 
Effectiveness  values  are  weighted  more  heavily  due  to  the  60/40  split  in  favor  of  System 
Effectiveness  at  the  top  level  of  the  hierarchy.  Figure  4  shows  a  graphical  comparison  of 
the  lowest  level  weightings.  The  values  of  Dependability ,  Understandability,  and 
Resiliency  are  on  the  same  tier  as  the  other  lowest-tier  values  but  each  is  further 
decomposed.  The  values  under  these  three  values  are  stacked  in  the  graphical 
presentation  to  allow  comparison  across  a  single  level  of  decomposition. 
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Table  4.  System  Effectiveness  Value  Definitions 
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Table  5.  Architecture  Quality  Value  Definitions 
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Figure  4.  Comparison  of  Value  Weights 
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4.2.  Develop  Evaluation  Measures 

Once  the  values  of  the  value  hierarchy  were  established,  each  of  the  lowest-level 
values  needed  at  least  one  measure.  The  research  team  developed  a  draft  set  of  measures 
and  presented  it  to  the  same  set  of  subject  matter  experts  and  SEIWG  members  as  the 
value  hierarchy.  These  measures  took  on  two  basic  forms;  they  either  looked  for  aspects 
of  the  architecture  that  added  value  or  they  looked  to  identify  aspects  of  the  architecture 
that  detracted  from  its  value.  An  aspect  that  would  add  value  might  be  the  inclusion  of 
critical  force  protection  concepts  such  as  a  threat  assessment  plan.  An  aspect  that  would 
detract  value  might  be  the  presence  of  unnecessary  or  duplicative  information. 

Discussion  on  the  draft  measures  resulted  in  numerous  modifications  as  well  as 
the  deletion  and  addition  of  measures.  The  discussion  also  included  the  information  that 
would  be  needed  to  score  the  measures  as  well  as  the  views  within  which  the  information 
would  be  contained;  the  views  that  the  group  identified  were  recorded  as  part  of  the 
measure.  Once  the  measures  were  complete,  the  weighting  of  the  completed  value 
hierarchy  was  revalidated.  The  complete  list  of  measures,  including  their  definitions  and 
locations  to  find  the  information  to  score  them,  can  be  found  in  Table  6  and  7  for  the 
Architectural  Quality  and  System  Effectiveness  branches,  respectively. 
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Using  the  resulting  Value-Driven  Enterprise  Architecture  (VDEA)  score  on  the 
Joint  Force  Protection  Advanced  Security  System  (JFPASS)  architecture  provided  a 
baseline  from  which  improvements  to  the  architecture  could  be  judged.  During  the 
analysis  of  the  architecture,  it  was  noted  that  some  of  the  measures  scored  poorly  because 
the  views  that  contain  the  information  needed  to  assess  the  measure  had  not  yet  been 
developed  (Cotton  &  Haase,  2009;  Mills,  2009).  The  absence  of  these  views  detracted 
from  the  overall  value  score  for  the  architecture,  which  shows  that  some  views  are 
particularly  valuable  to  the  decision-maker.  For  more  information  on  the  application  and 
analysis  of  the  VDEA  score  to  JFPASS  and  the  Information  and  Resource  Support 
System,  refer  to  Cotton  and  Haase  (2009)  and  Mills  (2009). 

4.3.  Evaluation  “Measures-by-Views”  Matrix 

The  VDEA  methodology  identified  36  evaluation  measures  from  the  value 
hierarchy  for  JFPASS  architecture.  Combining  those  36  evaluation  measures  with  the  29 
possible  views  from  the  Department  of  Defense  Architectural  Framework  (DoDAF) 
creates  a  matrix  with  1,044  relationships  between  views  and  evaluation  measures.  After 
initial  identification  of  relationships  between  measures  and  views,  this  matrix  can  be 
reduced  by  removing  non-discriminating  measures  and  views  that  are  not  related  to  the 
remaining  measures.  This  results  in  the  removal  of  nine  measures  leaving  a  total  of  27 
measures  that  are  used  in  this  evaluation. 

4.4.  Identifying  Relationships  and  Numerical  Descriptors 

The  process  of  identifying  relationships  was  based  on  the  findings  from  the 
VDEA  process,  which  identified  for  each  measure  the  required  views  that  would  provide 
the  information  needed  to  score  the  architecture.  The  view  identification  was  done  as 
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part  of  the  development  of  the  evaluation  measures  step  of  the  VDEA  process.  Many  of 
the  evaluation  measures  and  the  view  relationships  were  validated  during  the  scoring  of 
the  JFPASS  architecture  with  the  VDEA  process,  though  some  of  the  views  required  for 
evaluation  were  not  available  so  those  relationships  were  not  validated  (Cotton  &  Haase, 
2009;  Mills,  2009).  A  review  of  the  DoDAF  requirements  for  views  was  used  to  help 
validate  all  the  identified  relationships.  The  initial  identification  of  relationships  can  be 
seen  in  Table  8.  The  numerical  descriptors  of  the  strength  of  the  relationships  were 
assumed  linearly  additive  across  each  measure.  That  is  the  strength  of  the  relationship 
between  a  view  and  a  measure  is  one  over  the  total  number  of  views  associated  with  the 
measure. 

xt  =  —  (2) 

1  Mi  V  ; 

where  x,  is  the  strength  of  the  relationship  between  view  and  the  ith  measure  and  M,  is  the 
number  of  views  associated  with  the  ith  measure. 
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Table  8.  Initial  Evaluation  Measures  by  View  Matrix 


1-5 


4.4.1.  Non-Discriminating  Evaluation  Measures 

Several  of  the  evaluation  measures  concern  the  quality  of  the  existing  views  and 
are  not  associated  with  any  particular  view.  These  non-discriminating  measures  are  listed 
in  Table  9.  These  measures  differ  from  the  other  measures  because  instead  of  looking  for 
the  architecture  to  convey  some  specific  information,  they  seek  to  measure  the  quality  of 
existing  views;  therefore,  any  set  of  views  can  be  evaluated  against  these  measures.  This 
means  that  no  view  is  more  important  to  these  measures  than  any  other  view.  For 
example,  if  the  only  view  created  was  the  OV-1,  then  when  it  is  evaluated  for  the 
measure  connections  this  view  could  gain  all  available  value  for  that  measure.  Likewise, 
if  the  OV-5  is  the  only  view  created,  then  it  too  would  gain  full  value  under  that  measure. 
Since  these  non-discriminating  measures  are  equally  applicable  to  all  views,  they  are  not 
useful  in  discriminating  between  views.  For  this  reason,  they  are  not  included  in  the 
analysis. 


Table  9.  Non-Discriminating  Evaluation  Measures 


MEASURE 

Applicability 

Connections 

Measures  All  Available  Views 

Architecture  Redundancy 

Measures  All  Available  Views 

Architecture  Economy 

Measures  All  Available  Views 

OV  Readability 

Measures  All  Available  Operational  Views 

SV  Readability 

Measures  All  Available  System  Views 

Scale 

Measures  All  Available  Views 

DoDAF  Compliancy 

Measures  All  Available  Views 

Internal  Consistency 

Measures  All  Available  Views 

External  Consistency 

Measures  All  Available  Views 
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4.4.2.  Architectural  Quality  Evaluation  Measures 

The  non-discriminatory  measures  are  all  part  of  the  architectural  quality  branch  of 
the  value  hierarchy.  With  their  removal,  there  are  1 1  Architectural  Quality  Evaluation 
Measures  remaining;  these  are  summarized  with  their  architecture  view  relationships  in 
Table  10.  The  majority  of  these  measures  are  identified  as  being  related  to  the  AV-1. 

The  DoDAF  volume  II  (2007)  describes  the  AV-1  as  both  an  “executive  level  summary” 
and  a  “planning  guide”  for  architecture  development;  as  a  result,  it  makes  sense  that  most 
of  the  information  needed  to  determine  the  value  of  an  architecture  would  be  contained 
there.  For  instance,  the  involvement  of  subject  matter  experts  (SME)  was  determined  to 
contribute  to  the  value  of  an  architecture  and  is  represented  in  the  value  hierarchy  by  the 
value  Stakeholder  Involvement.  Stakeholder  Involvement  has  two  evaluation  measures, 
SME  EFFECTIVENESS  and  SME  INVOLVEMENT;  in  order  to  measure  it,  information  on  the 
number  of  SMEs,  their  branch  of  service,  and  years  of  experience  are  needed.  All  of  this 
information  can  be  included  in  an  AV-1  under  the  heading  of  Architecture  Project 
Identification. 
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Table  10.  Architecture  Quality  Evaluation  Measures  by  View  Matrix 


Source  Views 


MEASURE 

AV-1 

OV-1 

OV-2 

OV-3 

OV-4 

OV-5 

OV-6 

SV-2 

SV-5 

SV-7 

SV-8 

SV-9 

TV-1 

Access 

X 

Product  Locatability 

X 

Document  Protection 

X 

Access  Control 

X 

File  Management 

X 

File  Format 

X 

Tool  Format 

X 

Decomposition 

X 

Requirement  Traceability 

X 

SME  Effectiveness 

X 

SME  Involvement 

X 

The  evaluation  measures  of  DECOMPOSITION  and  REQUIREMENT  TRACEABILITY 
are  related  to  the  OV-5  and  SV-5,  respectively.  DECOMPOSITION  specifically  measures 
the  level  of  decomposition  in  the  OV-5.  REQUIREMENT  TRACEABILITY  is  measured  as  a 
percentage  of  requirements  that  are  traced  to  functions  in  an  SV-5.  The  DoDAF 
describes  three  types  of  SV-5,  designated  as  the  SV-5a  (Operational  Activity  to  System 
Functions  Traceability  Matrix),  the  SV-5b  (Operational  Activity-to-Sy stems  Traceability 
Matrix),  and  the  SV-5c  (Operational  Activity  to  Services  Traceability  Matrix).  For  the 
puiposes  of  measuring  TRACEABILITY,  all  three  versions  of  the  SV-5  are  capable  of 
displaying  the  necessary  information  so  they  are  considered  equivalent  and  referred  to 
collectively  as  SV-5. 

4.4.3.  System  Effectiveness  Evaluation  Measures 

The  remaining  16  Evaluation  Measures  fall  under  the  System  Effectiveness  branch 
and  were  found  to  be  related  to  a  total  of  13  views  as  shown  in  Table  11.  The  three 
variations  of  the  SV-5  are  considered  equivalent  for  the  purposes  of  providing 
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information  on  OPERATIONAL  NEEDS;  likewise,  the  three  versions  of  the  OV-6  are 
considered  equivalent  for  the  purposes  of  providing  information  on  SYSTEM 
REDUNDANCY. 

The  evaluation  measures  under  the  System  Effectiveness  branch  look  at  how  the 
system,  as  described  by  the  architecture,  meets  the  objectives  of  the  instantiated  system. 
The  purpose  being  that  the  system  has  to  1)  be  the  right  system  to  meet  those  objectives 
and  2)  be  described  in  sufficient  detail  to  show  how  it  will  meet  those  objectives.  As  a 
result,  the  views  associated  with  most  of  the  System  Effectiveness  evaluation  measures 
need  to  identify  specific  attributes  of  the  system  that  will  address  specific  objectives  and 
provide  sufficient  detail  to  show  how  it  will  address  that  objective.  The  following 
sections  will  discuss  each  measure  result  in  more  detail. 


Table  11.  System  Effectiveness  Evaluation  Measures  by  View  Matrix 


Source  Views 


MEASURES 

AV-1 

OV-1 

OV-2 

OV-3 

OV-4 

OV-5 

OV-6 

SV-2 

SV-5 

SV-7 

SV-8 

SV-9 

TV-1 

Operational  Needs 

X 

X 

X 

X 

mm 

mm 

Threat  Detection 

X 

X 

X 

Threat  Assessment 

X 

X 

X 

Warning  Plan 

X 

X 

X 

Technological  Availability 

mm 

Environmental  Impact 

mm 

Monetary  Practicality  -  Initial 

X 

Monetary  Practicality  - 
Maintenance 

■ 

■ 

■ 

■ 

X 

■ 

Adaptation 

mm 

Supportability  Requirements 

mm 

Reliability  Requirements 

mm 

System  Redundancy 

X 

Recoverability  Requirements 

X 

Joint  Operations 

X 

X 

X 

X 

mm 

NESI  Development 

mm 

NESI  Evaluation 

1  x  | 
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4.4.3.I.  OPERATIONAL  NEEDS 


Due  to  the  wide  range  of  possible  operational  needs  that  a  system  could  be 
designed  to  address,  the  views  required  to  describe  how  the  system  will  meet  those  needs 
can  vary.  For  the  JFPASS,  it  was  decided  during  the  development  of  the  evaluation 
measure  that  the  AV-1,  OV-1,  OV-3,  OV-5,  SV-5,  and  SV-7  were  the  pertinent  views  for 
describing  how  the  system  would  address  OPERATIONAL  NEEDS.  The  AV-1  provides  the 
scope  and  purpose  of  the  system.  The  three  operational  views  describe  the  system 
functionality  and  show  how  those  functions  relate  to  OPERATIONAL  NEEDS.  The  SV-5 
connects  the  system  functionality  from  the  operational  views  to  actual  system 
components.  The  SV-7  identifies  the  level  of  performance  that  each  system  component 
needs  to  achieve  to  fully  address  the  OPERATIONAL  NEEDS.  The  strength  of  each  of  the 
six  relationships  for  OPERATIONAL  NEEDS  was  described  as  one  divided  by  the  total 
number  of  relationships  or  0.167. 

4.4.3.2.  THREAT  DETECTION,  THREAT  ASSESSMENT,  and  WARNING  PLAN 

Three  of  the  key  aspects  of  a  joint  force  protection  system  are  threat  detection, 
threat  assessment,  and  providing  warning.  These  three  aspects  come  from  the  detect, 
assess,  warn,  defend,  and  recover  construct  (Protection  Assessment  Branch,  Joint  Staff, 
2004);  for  the  JFPASS,  it  was  decided  to  concentrate  on  the  first  three  aspects  of  that 
construct.  Similar  to  OPERATIONAL  NEEDS,  it  was  determined  that  the  OV-1,  OV-3,  and 
OV-5  were  the  appropriate  views  to  describe  the  system  functionality,  and  how  the 
system  would  address  the  three  key  concepts  of  detect,  assess,  and  warn.  Each  view  was 
given  equally  emphasis  resulting  in  the  strength  of  the  relationship  between  view  and 
measure  being  described  as  0.333. 
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4.4.33.  TECHNOLOGICAL  AVAILABILITY,  ADAPTATION,  and  SYSTEM 

REDUNDANCY 

TECHNOLOGY  AVAILABILITY  simply  looks  at  technology  readiness  levels.  The 
DoDAF  does  not  specifically  call  for  technology  readiness  levels  to  be  included  in  any 
particular  view.  The  SV-9  describes  all  of  the  emerging  and  forecasted  technology 
advancements  that  will  impact  the  system;  because  of  its  focus  on  developing 
technologies,  it  was  deemed  the  appropriate  place  for  information  pertaining  to 
technology  readiness  levels.  ADAPTATION  falls  under  the  value  of  FLEXIBILITY  and 
measures  how  well  the  system  adapts  to  changing  threats  and  is  associated  with  the  SV-8. 
SYSTEM  REDUNDANCY  falls  under  the  value  of  SURVIVABILITY,  and  measures  the 
amount  of  redundancy  in  the  system  and  is  associated  with  the  OV-6.  These  three 
measures  are  each  related  to  only  one  architecture  view;  as  a  result,  each  the  strength  of 
of  those  relationships  is  described  as  a  one  and  the  full  weight  of  the  measure  is  given  to 
the  corresponding  view. 

4.43.4.  ENVIRONMENTAL  IMPACT,  NESI  DEVELOPMENT,  and  NESI 

EVALUATION 

ENVIRONMENTAL  IMPACT,  NESI  DEVELOPMENT,  and  NESI  EVALUATION  are  all 
concerned  with  compliance  with  guidance  and  regulation.  They  are  all  associated  with 
the  TV-1  and  whether  or  not  the  appropriate  regulations  and  guidance  are  listed  there. 

All  three  of  these  views  are  related  solely  to  the  TV-1  with  a  description  of  one,  which 
transfers  the  weight  of  all  three  measures  to  the  TV-1. 

4.43.5.  MONETARY  PRACTICALITY  -  INITIAL  and  MAINTENANCE 

MONETARY  PRACTICALITY  -  INITIAL  compares  the  acquisition  cost  versus  the 

budgeted  amount.  Similarly,  MONETARY  PRACTICALITY  -  MAINTENANCE  compares  the 


51 


operation  and  maintenance  cost  of  the  system  with  the  amount  budgeted  for  that  purpose. 
The  OV-5  allows  for  inclusion  of  costing  data  for  activities.  Developing  costing  data  by 
operational  activity,  as  opposed  to  system  component,  may  not  be  the  preferred  method 
but  it  is  the  only  one  that  the  DoDAF  was  found  to  support.  As  a  result  of  how  the 
DoDAF  supports  costing  efforts,  MONETARY  PRACTICALITY  -  INITIAL  and 
MAINTENANCE  are  related  with  the  OV-5. 

4.43.6.  SUPPORTABILITY,  RELIABILITY,  and  RECOVERABILITY 

REQUIREMENTS 

SUPPORTABILITY,  RELIABILITY,  and  RECOVERABILITY  REQUIREMENTS  measure 
the  values  of  Supportability,  Reliability,  and  Recoverability  respectively.  Each  of  these 
three  measures  looks  for  the  identification  of  appropriate  system  requirements  in  the  SV- 
7.  As  a  result,  their  relationship  to  the  SV-7  is  described  as  a  one  and  the  SV-7  is 
credited  with  the  full  weight  of  the  three  measures. 

4.43.1.  JOINT  OPERATIONS 

JOINT  OPERATIONS  measures  the  value  of  Interchangeability  by  verifying  the 
extent  to  which  the  system  described  in  the  architecture  accounts  for  the  various  services 
and  fits  them  into  a  joint  concept  of  operations.  Accounting  for  all  services  begins  with 
the  AV-1  as  it  provides  the  architect  with  the  ability  to  describe  the  scope  and  context  of 
the  architecture.  The  OV-2  describes  how  nodes  from  different  services  connect  and  the 
OV-3  elaborates  on  the  attributes  of  the  information  passed  between  nodes.  Both  of  these 
operational  views  are  important  for  identifying  the  important  nodes  within  each  service 
and  ensuring  they  are  appropriately  connected  within  the  system.  The  OV-4  describes  at 
a  higher  level  how  organizations  will  relate  to  each  other  and  the  roles  they  will  fill  in  the 
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system.  The  roles  each  service  and  its  subunits  will  fill  are  vitally  important  in 
accounting  for  all  services.  Lastly,  the  SV-2  is  needed  to  ensure  that  the  systems  that  are 
specific  to  each  service  are  capable  of  communicating  with  each  other  and  fulfilling  the 
value  of  Interchangeability. 

4.5.  Completed  Matrix 

Once  the  matrix  was  completed,  there  were  several  views  that  were  found  to  not 
be  associated  with  any  of  the  measures  and  therefore  are  not  important  to  the  decision¬ 
maker’s  values.  These  views  were  removed  from  the  matrix.  Not  counting  variations  of 
the  same  view,  for  instance  SV-4a  and  SV-4b,  9  views  were  removed  from  the  matrix 
leaving  a  total  of  13  views.  Additionally,  the  non-discriminating  measures  can  be 
removed  as  was  described  in  Section  4.4.1.  The  removal  of  the  non-associated  views  and 
non-discriminating  measures  leaves  a  matrix  of  27  measures  by  13  views;  this  is 
demonstrated  by  Table  12.  The  completed  matrix  is  presented  in  Table  13.  Each 
relationship  was  multiplied  by  the  global  weight  for  that  measure,  which  is  found  along 
the  left  side  of  the  matrix,  and  summed  for  each  view  to  obtain  a  total  score.  The  total 
score  for  each  view  is  found  at  the  bottom  of  the  matrix.  It  should  be  noted  that  due  to 
the  subtraction  of  the  non-discriminating  measures  from  the  analysis,  the  total  of  all  the 
global  weights  for  the  VDEA  measures  is  only  0.828. 
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Table  13.  Completed  Evaluation  Measure  by  View  Matrix 
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Weighted  Total  0.1964  0.0478  0.0066  0.0544  0.0066  0.1118  0.0400  0.0066  0.0268  0.1318  0.0270  0.0200  0.1520 


4.5.1.  View  Analysis 

Under  previously  stated  assumptions,  the  score  that  each  view  receives  is  a 
comparative  score  of  discriminating  importance,  meaning  that  a  view  with  a  larger  score 
is  more  important  than  a  view  with  a  smaller  score.  A  total  of  13  views  received  a  non¬ 
zero  score,  the  ranking  of  these  views  in  decreasing  importance  can  be  found  in  Table  14. 
These  13  views  represent  the  most  important  views  for  the  JFPASS  project.  These  are 
the  views  that  should  be  built  to  meet  the  overall  objective  of  the  JFPASS  architecture. 

Of  these  13  views,  only  nine  are  required  by  the  Joint  Capabilities  Integration  and 
Development  System  (JCIDS).  The  four  views  not  required  by  the  JCIDS  are  the  OV-2, 
OV-3,  SV-7,  and  SV-8.  Additionally,  two  views  required  by  the  JCIDS,  the  SV-4  and 
the  SV-6,  are  not  important  for  achieving  the  objective  of  an  architecture  for  the  JFPASS. 
Three  views  that  are  required  “as  applicable”  by  the  JCIDS  are  not  required  for  a  JFPASS 
architecture. 
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Table  14.  Rank  Order  of  Most  Important  Views 

JCIDS 


Rank  View  Importance  Required 


1 

AV-1 

0.1964 

Yes 

2 

TV-1 

0.1520 

Yes 

3 

SV-7 

0.1318 

No 

4 

OV-5 

0.1118 

Yes 

5 

OV-3 

0.0544 

No 

6 

OV-1 

0.0478 

Yes 

7 

OV-6 

0.0400 

Yes 

8 

SV-8 

0.0270 

No 

9 

SV-5 

0.0268 

Yes 

10 

SV-9 

0.0200 

No 

11 

OV-2 

0.0066 

Yes 

12 

OV-4 

0.0066 

Yes 

13 

SV-2 

0.0066 

Yes 

JCIDS  Required  Views  Not  Listed: 
SV-4,  SV-6,  SV-11,  OV-7,  TV-2 


4.5.2.  Build  Sequence  Analysis 

The  final  phase  of  analysis  is  solving  the  network  diagram  from  the  DoDAF 
version  1  deskbook  (2003)  given  the  list  of  most  important  views.  The  objective  of  this 
analysis  is  to  provide  an  ordered  list  for  view  creation  that  increases  the  usefulness  of  the 
architecture  as  quickly  as  possible  while  maintaining  the  advantages  of  following  the 
network  diagram.  Figure  5  is  a  simplified  version  of  the  network  diagram  suggested  in 
the  DoDAF  version  1  deskbook  (2003).  This  simplified  network  diagram  eliminates 
views  that  are  not  important  to  the  JFPASS  architecture  and  not  a  prerequisite  for  a  view 
that  is  important  to  the  architecture. 
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Figure  5.  Simplified  Network  Diagram  for  JFPASS  Architecture 


The  simplified  network  diagram  includes  15  views,  two  more  than  the  13  views 
found  to  be  important  to  a  JFPASS  architecture.  The  TV-2  is  not  required  for  any 
evaluation  measure  but  is  recommended  for  its  relationship  to  the  SV-8.  The  SV-1  is  also 
not  required  for  any  evaluation  measure;  however,  it  is  recommend  for  its  usefulness  in 
identifying  technical  standards  for  the  TV-1,  performance  standards  for  the  SV-7,  and  its 
relationship  to  the  SV-2. 

The  first  heuristic  applied  to  the  simplified  network  diagram  was  the  “steepest 
ascent”  heuristic,  which  selects  views  based  on  importance.  For  the  purposes  of  this 
analysis,  the  SV-1  is  considered  the  14th  most  important  view  and  the  TV-2  the  15th 
most  important  view.  The  build  sequence  that  this  “steepest  ascent”  heuristic  produced 
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resulted  in  both  the  TV-1  and  SV-7,  which  were  ranked  second  and  third,  respectively, 


being  built  late  in  the  sequence.  This  is  because  one  of  the  prerequisites  for  both  of  these 
views  is  the  SV-1,  which  is  ranked  near  the  bottom  of  the  views  being  considered.  The 
resulting  build  sequence  from  this  heuristic  can  be  seen  in  Table  15. 


Table  15.  Steepest  Ascent  Build  Sequence 


#  of 

Views 

Steepest  Ascent  Heuristic 

View 

Added 

Importance 

Cumulative 

Importance 

1 

AV-1 

0.1964 

0.1964 

2 

OV-1 

0.0478 

0.2442 

3 

OV-5 

0.1118 

0.3560 

4 

OV-6 

0.0400 

0.3960 

5 

SV-5 

0.0268 

0.4228 

6 

SV-9 

0.0200 

0.4428 

7 

OV-2 

0.0066 

0.4494 

8 

OV-3 

0.0544 

0.5038 

9 

OV-4 

0.0066 

0.5104 

10 

SV-1 

0.0000 

0.5104 

11 

TV-1 

0.1520 

0.6624 

12 

SV-7 

0.1318 

0.7942 

13 

SV-2 

0.0066 

0.8008 

14 

TV-2 

0.0000 

0.8008 

15 

SV-8 

0.0270 

0.8278 
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The  “most  successors”  heuristic  suggests  the  SV-1  earlier  in  the  build  sequence 


because  of  the  number  of  successor  views  to  the  SV-1.  This  allows  the  TV-1  and  SV-7  to 
move  up  in  the  build  sequence  but  the  SV-7  is  delayed  because  it  has  no  followers.  The 
resulting  build  sequence  from  the  application  of  this  heuristic  can  be  seen  in  Table  16. 


Table  16.  Most  Successors  Build  Sequence 


#  of 

Views 

Most  Critical  Successors  Heuristic 

Solution 

Number  of  Critical 

Successors 

Added 

Importance 

Cumulative 

Importance 

1 

AV-1 

Critical 

0.1964 

0.1964 

2 

OV-1 

3 

0.0478 

0.2442 

3 

OV-5 

Critical 

0.1118 

0.3560 

4 

SV-5 

2 

0.0268 

0.3828 

5 

OV-2 

2 

0.0066 

0.3894 

6 

SV-1 

2 

0.0000 

0.3894 

7 

TV-1 

Critical 

0.1520 

0.5414 

8 

SV-7 

Critical 

0.1318 

0.6732 

9 

OV-3 

0 

0.0544 

0.7276 

10 

OV-6 

0 

0.0400 

0.7676 

11 

SV-9 

0 

0.0200 

0.7876 

12 

OV-4 

0 

0.0066 

0.7942 

13 

SV-2 

0 

0.0066 

0.8008 

14 

TV-2 

0 

0.0000 

0.8008 

15 

SV-8 

0 

0.0270 

0.8278 

The  final  solution  used  the  “most  critical  followers”  heuristic.  For  this 
application,  the  top  four  views  by  importance  ranking  were  designated  as  critical.  The 
distinction  was  made  between  the  fourth  and  fifth  view  because  of  the  significant  drop  in 
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importance.  This  rule  selects  critical  views  first,  views  with  the  most  critical  followers 


second,  and  then  views  without  critical  followers  in  order  of  importance  score  third.  This 
solution  moved  up  the  SV-1  because  it  has  two  critical  followers  and  moved  up  the  TV-1 
and  SV-7  because  they  are  considered  critical.  This  solution  is  shown  in  Table  17. 


Table  17.  Most  Critical  Followers  Heuristic 


#  of 

Views 

Most  Successors  Heuristic 

Solution 

Number  of 

Successors 

Added 

Importance 

Cumulative 

Importance 

1 

AV-1 

14 

0.1964 

0.1964 

2 

OV-1 

13 

0.0478 

0.2442 

3 

OV-5 

10 

0.1118 

0.3560 

4 

OV-2 

7 

0.0066 

0.3626 

5 

SV-5 

5 

0.0268 

0.3894 

6 

SV-1 

4 

0.0000 

0.3894 

7 

SV-9 

2 

0.0200 

0.4094 

8 

TV-1 

1 

0.1520 

0.5614 

9 

TV-2 

1 

0.0000 

0.5614 

10 

SV-7 

0 

0.1318 

0.6932 

11 

OV-3 

0 

0.0544 

0.7476 

12 

OV-6 

0 

0.0400 

0.7876 

13 

SV-8 

0 

0.0270 

0.8146 

14 

OV-4 

0 

0.0066 

0.8212 

15 

SV-2 

0 

0.0066 

0.8278 

The  SV-1  is  a  critical  hinge  point  in  the  network  due  to  the  number  of  successors 
and  the  importance  of  those  successors.  The  SV-1  itself  does  not  add  to  the  growth  of  the 
architecture  but  unlocks  several  key  views  that  allow  for  rapid  growth.  The  major 
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difference  between  the  three  solutions  presented  here  is  the  placement  of  the  SV-1  in  the 
build  sequence.  Placing  the  SV-1  early  in  the  build  sequence  sacrifices  short  term  growth 
for  long  term  growth  as  is  the  case  with  the  “most  critical  followers”  heuristic.  All  three 
solutions  reach  the  same  end  point  but  have  different  levels  of  maturity  at  various  points 
in  development,  as  can  be  seen  in  Figure  6. 


Figure  6.  Cumulative  Progress  as  a  Function  of  Number  of  Completed  Views 
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Chapter  5.  Conclusion 


This  research  has  presented  a  methodology  to  focus  enterprise  architecture  view 
generation  on  the  stated  objectives  of  the  architecture  development  project.  The 
methodology  balances  values,  identifies  the  views  that  are  important  to  those  values,  and 
helps  the  program  manager  develop  architecture  in  a  logical  manner.  The  methodology 
also  answers  several  specific  questions  about  the  Department  of  Defense  Architectural 
Framework  (DoDAF)  and  the  Joint  Capabilities  Integration  and  Development  System 
(JCIDS)  process,  opening  up  avenues  for  future  research  that  will  assist  in  refocusing 
both  the  architectural  framework  and  the  system  acquisition  process  on  the  creation  of 
value. 

5.1.  Answers  to  Research  Question 

Aside  from  demonstrating  a  useful  program  management  tool,  this  research 
sought  to  examine  how  both  the  DoDAF  and  the  JCIDS  contribute  to  meeting  the  overall 
objective  of  an  architecture  development  project.  This  research  answers  questions  about 
how  the  DoDAF  views  contribute  to  the  Joint  Force  Protection  Advanced  Security 
System  (JFPASS)  architecture  and  how  the  JCIDS  requirements  compare  with  what  is 
important  to  the  JFPASS  architecture.  This  section  details  the  answers  to  each  research 
question  introduced  in  Chapter  1.  The  first  two  research  questions  focused  on  the 
importance  of  individual  DoDAF  views  and  how  they  contribute  to  the  architecture. 
Questions  three  and  four  focused  on  the  JCIDS  requirements  for  architecture  and  how 
those  requirements  fit  with  the  findings  from  questions  one  and  two.  The  final  question 
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demonstrated  how  the  results  of  the  methodology  could  be  used  to  support  decision 


making  in  an  architecture  development  effort. 

5.1.1.  What  DoDAF  views  are  the  most  important  to  the  JFPASS  architecture? 

The  analysis  of  the  results  shows  that  there  are  13  views  that  are  more  important 
than  the  other  DoDAF  views  for  the  JFPASS  architecture.  All  of  the  views  described  in 
the  DoDAF  have  the  potential  of  conveying  useful  information  about  a  system;  however, 
the  13  most  important  views  listed  in  Table  18  are  the  ones  best  suited  for  conveying  the 
information  that  subject  matter  experts  and  Security  Equipment  Integration  Working 
Group  (SEIWG)  officials  value  most.  Of  these  13  views,  the  top  four  views  are 
significantly  more  important  than  the  remaining  nine  views. 


Table  18.  The  Most  Important  Views  for  the  JFPASS 


Rank 

View 

Importance 

1 

AV-1 

0.1964 

2 

TV-1 

0.1520 

3 

SV-7 

0.1318 

4 

OV-5 

0.1118 

5 

OV-3 

0.0544 

6 

OV-1 

0.0478 

7 

OV-6 

0.0400 

8 

SV-8 

0.0270 

9 

SV-5 

0.0268 

10 

SV-9 

0.0200 

11 

OV-2 

0.0066 

12 

OV-4 

0.0066 

13 

SV-2 

0.0066 
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The  most  important  view  to  the  JFPASS  architecture  is  the  AV-1;  this  view  is 
23%  more  important  than  the  second  most  important  view.  The  importance  of  the  AV-1 
stems  in  large  part  to  its  flexibility  in  conveying  a  wide  variety  of  information  to  the 
potential  users  of  an  architecture.  The  ability  of  the  AV-1  to  identify  the  scope  and 
purpose  of  the  architecture  provides  useful  information  on  how  to  interpret  the  other 
views.  The  AV-1  has  the  capability  to  list  the  people  and  offices  involved  in  the  creation 
of  the  architecture  which,  when  used  to  describe  the  involvement  of  subject  matter 
experts  and  stakeholders  from  across  the  four  services,  adds  credibility  to  the 
architecture.  The  AV-1  can  also  provide  potential  users  with  information  on  gaining 
access  to  and  using  the  architecture,  such  as  any  applicable  protections  and  controls, 
formatting,  and  software  tools.  Given  the  powerful  and  flexible  format  of  the  AV-1  as  an 
“executive  summary”  and  “planning  guide”  (Department  of  Defense,  2007)  for  the 
creation  of  an  architecture,  it  is  not  surprising  that  it  was  found  to  be  extremely  important 
for  the  JFPASS. 

The  second  and  third  most  important  views  to  a  JFPASS  architecture  are  the  TV-1 
and  the  SV-7.  The  ability  of  the  system  to  operate  in  a  variety  of  environments  and 
locations  as  well  as  interoperate  with  other  services  requires  it  to  conform  to  a  number  of 
technical  standards.  Additionally,  as  with  most  systems,  the  JFPASS  must  meet  a 
number  of  operational  needs  as  well  as  be  durable  and  easy  to  maintain.  The  simplest 
way  to  ensure  that  the  system  eventually  meets  technical  and  performance  standards  is  to 
identify  those  standards  as  early  and  explicitly  as  possible.  The  TV-1  and  SV-7  provide 
the  architecture  the  capability  of  explicitly  stating  the  technical  and  performance 
standards  at  the  outset  of  the  acquisition  process. 
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The  fourth  most  important  view  is  the  OV-5  Operational  Activity  Model.  The 
purpose  of  any  system  is  to  fulfill  an  operational  need  by  performing  some  task.  How  the 
system  is  to  perform  the  task  is  important  to  the  user  and  the  designer.  The  OV-5 
provides  a  format  for  detailing  how  the  system  will  perform  its  given  function  and  meet 
the  operational  need.  Most  of  the  importance  of  the  OV-5  comes  from  its  link  to  the 
measures  under  the  value  of  Purposefulness  and  the  relatively  high  weighting  of  that 
value.  The  OV-5  also  gains  some  value  under  the  two  measures  of  monetary  practicality 
as  the  only  place  in  the  DoDAF  that  supports  the  inclusion  of  costing  data.  Providing 
costing  data  by  activity  may  not  be  ideal  for  all  systems,  and  this  may  be  an  area  that  the 
DoDAF  could  be  improved. 

The  remaining  nine  views  from  the  top  13  account  for  approximately  28%  of  the 
total  importance  of  the  architecture.  These  nine  views  contribute  significantly  to  the 
achievement  of  the  overall  objective  for  the  architecture  and  should  be  created  to  ensure 
the  full  achievement  of  that  overall  objective.  However,  individually  they  do  not  warrant 
detailed  discussion  here. 

5.1.2.  What  DoDAF  views  should  be  built  based  on  the  overall  objective  of  the 
JFPASS  architecture? 

An  analysis  of  each  evaluation  measure  showed  that  there  were  13  views  of 
greater  importance  than  the  other  DoDAF  views.  These  13  views  cover  all  of  the 
evaluation  measures  being  considered.  Their  ability  to  completely  cover  the  evaluation 
measures  means  that  these  13  views  are  capable  of  gaining  full  value  for  the  JFPASS 
architecture  when  evaluated  with  the  VDEA  score.  A  sub-set  of  these  13  views  may  also 
be  able  to  gain  full  value  but  no  more  than  these  13  is  required. 
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5.1.3.  Which  if  any  JCIDS  required  views  are  emphasized  by  the  values 
associated  with  the  JFPASS  architecture? 

The  JCIDS  process  requires  the  SV-4  and  the  SV-6  for  both  the  Milestone  B  and 
Milestone  C  decision  points.  The  OV-7,  SV-11,  and  the  TV-2  are  required  as  applicable 
at  different  milestone  decision  points.  None  of  these  views  was  found  particularly 
beneficial  to  a  JFPASS  architecture.  This  suggests  some  level  of  disconnect  between  the 
values  of  subject  matter  experts  and  the  program  office  with  the  JCIDS  process.  This 
research  is  unable  to  examine  the  JCIDS  process  to  identify  the  purpose  of  requiring 
these  particular  views,  but  these  findings  suggest  it  may  be  beneficial  to  examine  JCIDS 
requirements  and  subject  matter  expert  assumptions  for  architecture. 

5.1.4.  Which  if  any  views  that  are  important  to  the  JFPASS  architecture  are  not 
required  by  JCIDS? 

Four  of  the  13  views  that  were  found  to  be  important  to  a  JFPASS  architecture  are 
not  required  for  the  JCIDS  process.  These  views  are  the  SV-7,  OV-3,  SV-8,  and  SV-9. 
The  SV-7  is  by  far  the  most  important  of  the  four  because  of  its  ability  to  clearly  lay  out 
performance  requirements  for  the  system  to  be  designed  around.  The  SV-7  on  its  own 
accounts  for  approximately  16%  of  the  importance  of  the  13  views;  this  represents  a 
significant  disconnect  with  the  JCIDS.  The  remaining  three  views,  OV-3,  SV-8,  and  SV- 
9,  account  for  approximately  12%  of  the  importance  of  the  13  views,  thereby  making 
them  an  important  combined  contribution  to  the  architecture.  The  JCIDS  process  does 
not  require  these  views  but  also  does  not  directly  prohibit  their  creation  either.  However, 
by  establishing  a  set  of  required  views,  the  JCIDS  process  tends  to  drive  a  focus  on  the 
required  views  that  may  limit  the  resources  available  for  the  creation  of  non-required 
views. 
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5.1.5.  Based  on  the  suggested  network  diagram  from  the  DoDAF  Deskbook 
(Department  of  Defense,  2003),  in  what  order  should  the  views  be  created  to 
most  rapidly  increase  the  usefulness  of  the  JFPASS  Architecture? 

A  simple  heuristic  for  deciding  the  order  in  which  to  create  the  views  is  to  simply 
create  the  most  important  view  for  which  all  prerequisite  views  have  been  created.  The 
drawback  to  this  “steepest  ascent”  approach  is  that  the  resulting  build  process  can  delay 
creation  of  low  importance  views  with  high  importance  successors.  This  can 
dramatically  delay  the  creation  of  high  importance  views,  as  is  the  case  with  the 
application  of  the  heuristic  in  this  research.  A  better  solution  accounts  for  those  higher 
importance  views  that  are  towards  the  end  of  the  build  sequence  and  can  dramatically 
improve  the  resulting  growth  curve  as  seen  in  Figure  7.  However,  regardless  of  the  exact 
order  in  which  the  views  are  created,  if  all  13  of  the  recommend  views  are  created  then 
the  full  value  will  be  obtained.  Both  the  relative  importance  of  each  view  and  the 
suggested  network  diagram  are  important  tools  for  guiding  architectural  development. 

The  program  objectives  for  growth  over  time  and  resource  constraints  will  determine 
which  build  sequence  is  most  suitable.  The  selection  of  the  most  appropriate  build 
sequence  will  also  need  to  account  for  the  number  of  views  that  can  be  created.  If  time  or 
other  resources  constrain  the  total  number  of  views  that  can  be  created,  then  the  objective 
would  be  to  optimize  the  solution  for  that  number  of  views.  For  instance,  if  only  nine 
views  can  be  created  under  given  funding  constraints,  then  the  “most  critical  followers” 
heuristic  provides  the  best  solution  of  the  three  examined  here.  If  only  five  views  can  be 
created,  then  the  “steepest  ascent”  heuristic  provides  the  better  solution. 
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In  the  case  of  the  JFPASS,  architecture  view  creation  has  already  begun  with  a 
number  of  views  having  been  created  already.  The  VDEA  score  methodology  was 
applied  to  the  JFPASS  views  listed  in  Table  19  by  Cotton  and  Haase  (2009)  and  Mills 
(2009).  In  the  case  of  an  architecture  project  under  development  the  build  sequence 
analysis  can  still  be  applied.  Views  should  still  be  created  in  the  order  prescribed  by  the 
chosen  build  sequence,  if  a  view  from  the  build  sequence  has  already  been  created  it 
should  be  evaluated  using  the  VDEA  score  methodology  and  any  deficiencies  corrected. 
Based  on  the  current  status  of  the  JFPASS  architecture  and  the  build  sequence  solution 
provided  from  the  “most  critical  followers”  heuristic,  shown  in  Table  20,  the  next  task  for 
the  JFPASS  architecture  is  to  revise  the  AV-1  followed  by  the  OV-5,  than  architecture 
development  can  proceed  in  the  order  prescribed  by  the  build  sequence  omitting  any 
views  already  completed. 
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Table  19.  Completed  JFPASS  Views  (Mills,  2009) 


AV-1 

SV-1 

OV-1 

SV-2 

OV-2 

SV-4 

OV-4 

SV-6 

OV-5 

TV-1 

OV-6c 

Table  20.  Most  Critical  Successors  Heuristic  with  JFPASS  Current  Status 


#  of 

Views 

Most  Critical  Successors  Heuristic 

Sequence 

Number  of  Critical 

Successors 

Added 

Importance 

Cumulative 

Importance 

JFPASS  Current  Status 

1 

AV-1 

Critical 

0.1964 

0.1964 

Draft,  needs  revision 

2 

OV-1 

3 

0.0478 

0.2442 

Complete 

3 

OV-5 

Critical 

0.1118 

0.3560 

Draft,  needs  revision 

4 

SV-5 

2 

0.0268 

0.3828 

None 

5 

OV-2 

2 

0.0066 

0.3894 

Complete 

6 

SV-1 

2 

0.0000 

0.3894 

Complete 

7 

TV-1 

Critical 

0.1520 

0.5414 

Complete 

8 

SV-7 

Critical 

0.1318 

0.6732 

None 

9 

OV-3 

0 

0.0544 

0.7276 

None 

10 

OV-6 

0 

0.0400 

0.7676 

Complete  (OV-6c) 

11 

SV-9 

0 

0.0200 

0.7876 

None 

12 

OV-4 

0 

0.0066 

0.7942 

Complete 

13 

SV-2 

0 

0.0066 

0.8008 

Complete 

14 

TV-2 

0 

0.0000 

0.8008 

None 

15 

SV-8 

0 

0.0270 

0.8278 

None 
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5.3.  Methodology  Strengths 

The  VDEA  Development  Goals  (VDEA-DG)  methodology  provides  a  useful  tool 
in  planning  and  managing  architecture  development  that  focuses  efforts  the  type  and 
order  of  importance  of  architecture  view  development.  In  combination  with  the  VDEA 
score,  this  methodology  provides  a  comprehensive  tool  that  explicitly  identifies  the 
architecture  objectives,  aids  in  selecting  and  prioritizing  view  creation,  and  tracks 
progress  toward  a  complete  value-driven  architecture. 

5.4.  Methodology  Weaknesses 

Though  the  methodology  presented  here  holds  great  potential  for  further 
application  in  the  management  of  architectural  development,  there  are  areas  that  need 
further  refinement.  The  current  process  of  linking  views  to  value  measures  is  not 
rigorously  developed.  The  identification  of  views  was  discussed  with  subject  matter 
experts  in  architecture  and  with  SEIWG  members  in  the  context  of  scoring  the 
architecture  with  the  VDEA  score.  In  future  applications,  this  discussion  should  take 
place  with  an  understanding  of  the  impact  it  will  have  on  view  selection. 

This  research  used  the  global  weights  of  the  evaluation  measures  and  the  linkage 
between  measures  and  views  as  a  proxy  for  the  importance  of  each  view.  This 
methodology  does  not  take  into  account  the  interdependency  of  views  for  meeting  a 
measure  or  the  ability  of  multiple  views  to  convey  the  information  necessary  for  a 
particular  evaluation  measure.  As  a  result,  the  actual  value  gained  by  creating  a  view 
cannot  be  evaluated  in  order  to  create  maximum  value  with  a  minimum  number  of  views. 

The  methodology  used  to  answer  this  research  question  assumed  that  no  views 
had  been  previously  developed,  as  in  an  architectural  effort  that  has  not  yet  begun.  In  the 
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case  of  the  JFPASS,  several  views  have  already  been  built,  in  some  cases  without  their 
prerequisites  having  been  built.  In  a  case  such  as  this,  it  is  recommended  to  build  any 
missing  prerequisites  for  views  that  have  already  been  built  and  update  the  rest  of  the 
architecture  as  necessary.  Then  the  build  sequence  can  be  solved  for  the  remaining 
views. 

5.5.  Recommended  Future  Research 

Further  research  should  explore  the  possibility  of  extending  the  methodology 
beyond  the  use  as  a  proxy  for  importance  and  look  specifically  at  the  abilities  of  each 
view  to  generate  value  under  different  measures.  This  research  should  take  into  account 
the  interdependencies  of  views  and  identify  the  value  of  single  views  and  groups  of 
views.  This  can  be  accomplished  by  considering  the  single  dimensional  value  function 
(SDVF)  when  connecting  views  to  measures.  Inclusion  of  the  SDVF  will  also  allow  the 
identification  of  the  minimum  views  for  creating  full  value.  Additionally,  the  heuristics 
used  for  solving  the  build  sequence  are  rudimentary  and  better  approaches  may  exist.  A 
methodology  such  as  simulated  annealing  (Kirkpatrick,  Gelatt,  &  Vecchi,  1983)  or 
combinatorial  optimization  (Cook,  Cunningham,  Pulleyblank,  &  Schrijver,  1997)  may  be 
applied  to  better  identify  the  optimal  solution. 

This  research  identified  areas  where  the  DoDAF  lacks  support  for  information 
areas  that  are  of  value  to  the  JFPASS  architecture  program.  The  evaluation  measures  of 
MONETARY  PRACTICALITY  -  INITIAL  and  MAINTENANCE  required  costing  data  for 
evaluating.  After  examining  the  DoDAF  for  references  to  costing  data,  the  OV-5  was  the 
only  view  found  to  support  the  inclusion  of  costing  data  and  then  simply  as  an  activity 
cost  estimate.  MONETARY  practicality  was  found  to  be  an  important  aspect  of  the 
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JFPASS  that  needed  to  be  captured  in  the  architecture.  The  importance  of  MONETARY 
PRACTICALITY  seems  logical,  given  the  important  role  cost  and  budget  play  in  decision¬ 
making.  Further  research  should  be  done  to  find  areas  where  the  DoDAF  can  be  refined 
and  developed  to  improve  support  for  valuable  information  by  supporting  areas  of 
interest  such  as  monetary  practicality. 

Concerns  were  also  raised  as  to  how  the  JCIDS  requirements  for  architecture 
views  could  hinder  creation  of  architecture  views  to  meet  the  overall  objective,  as 
identified  by  the  value  hierarchy.  Some  disconnects  were  found  between  the  JCIDS 
requirements  and  the  views  found  to  be  important  to  the  JFPASS  architecture.  The  major 
finding  was  the  absence  of  the  SV-7  from  the  JCIDS  requirements  for  milestone 
decisions.  This  view  was  found  to  be  extremely  important  to  the  architecture  and  was 
linked  to  the  values  under  maintainability,  which  was  of  particular  interest  to  force 
protection  experts.  The  identification  of  performance  requirements  is  an  important  early 
step  in  designing  any  system  and  the  SV-7  is  designed  for  this  purpose;  its  absence  from 
the  JCIDS  and  milestone  decision  making  is  surprising.  The  OV-3,  SV-8,  and  SV-9  were 
also  found  to  be  important  to  the  JFPASS  architecture  but  are  not  included  in  the  JCIDS 
requirements,  whether  these  views  would  also  be  important  to  other  architectures  is 
difficult  to  assess.  Further  research  into  the  JCIDS  support  for  value-driven  architecture 
by  applying  this  methodology  to  other  architecture  projects  would  show  what  trends  exist 
in  values  for  architecture,  the  importance  of  individual  views,  and  how  the  JCIDS 
requirements  align  with  identified  trends.  This  information  could  then  be  used  to  justify 
refinement  of  the  JCIDS  to  allow  view  selection  based  on  a  value  hierarchy. 
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